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ABSTRACT 



This thesis describes an experiment conducted at the Naval 
Postgraduate School (UPS) during the period 15 October 19 (> 5 
through 20 October 1985. Specifically, the experiment evaluates 
"pull-down window" microcomputer technology, continuous speeon 
recognition equipment, and standard computer keyboard entry to 
input commands in a command and control environment. Using the 
Naval Warfare Interactive Simulation System (NVJISS) as a controlled 
medium, military problems were posed to test subjects in specific 
light and noise environments. 

Although the results are not entirely conclusive, they do 
demonstrate a distinct advantage in using continuous speech or 
keyboard entry modes over the drop-down window technology of the 
Macintosh (if subject training time is not a significant restric- 
tion). Either the continuous speech, or the keyboard method was 
clearly superior in all environments. 
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I. INTRODUCTION 



A. OVERVIEW 

Computer Technology has ircreased exponentially ever trie past 
several years. As a result, work is now being completed more 
efficiently and inexpensively than ever before. Nevertheless, 
faced with a myriad of available computer options, it behooves 
the individual user to choose one that is not only user frienoly, 
but is both cost effective and a proven performer. Time has 
always been a critical factor for any business, and people are 
now realizing that using a computer only constitutes the first 
step toward achieving reductions ir. this area. 

In recent years there has been a movement afoot to simplify 
the all important man-machine interface which counts so heavily 
in on-line efficiency computations. While the computer keyboard 
has long been the industry standard, its speed is severely limited 
by the skill/training of the user; and the subject's knowledge of 
the particular command keys. Due primarily to these reccgnizec 
limitations, more attention is being paid to a faster, and r. ore 
sophisticated means of communication between user ana machine: 
Automatic Speech Recognition (ASR). This concept was first introduced 
in the late 1950's, and significant progress was made within trie 
last 15 years. It is known that free speech, in its natural 
form, is the fastest and most economical means for homo sapiens 
to commun i c a t e amongst themselves. Applying this fact to the 
user-machir.e interface dilemma, it has been shown that machines 
can not only be taught to resporia to any language, but that 
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speech for this type of communicat ion leaves the user's hands ano 
feet free in order to perforir important concurrent functions 
(i.e., viewing Graphics screens or decision support aids, personnel 
interface, etc.). While admittedly the progress toward an eccnor.- 
ically feasible voice recognition system has been slew, A SR 
proaucts are now making themselves known in industrial, military 
and security-oriented applications. It is this encouraging trend, 
coupled with the significant accomplishments in each area of use 
that leads researchers to believe in its unlimited potential. 
[Ref. 1 and 2] 

The newest development in interface technology is the "pull-down 
window" incorporated in the Macintosh microcomputer. (See Enclosures 
1-6 in Appendix G.) This methodology, in conjunction with the use 
of a mouse peripheral input device, presents a " rectangular-s ha pec. 
menu" on a computer screen within which the user writes and reads 
data representing a program in the computer. Because the mover ent 
of the cursor matches the movement of the mouse, the user need 
only place the cursor on the oesired command and press a conveniently 
located button for instant activation. Although no comparisons 
(individually or collectively) have currently been made with the 
more traditional interface devices because of its recent emergence 
in the marketplace, its ease of use and efficient operation bear 
true witness to its untapped possibilities. [Ref. 3] 

This thesis is concerned with evaluating the speed, accuracy 
and efficiency of the three previously described input devices. 
The data gathered for analysis was generated by subjects 
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participating in an experiment using the Naval Warfare Interactive 
Simulation System (HWISS), and will be discus sea in detail in 
later sections. 

B. SCOPE OF THE THESIS 

Although numerous experiinerts concerning automatic speech 
recognition have been conducted at the Naval Postgraduate Scnoo_ 
(NFS) , Monterey, California, only one previous experiment (conducted 
by Wright and Hanson; two NPS graduate students) has attemptec a 
comprehensive evaluation of computer input technologies ir a 
military command and control environment. That experiment was the 
first attempt to evaluate two voice recognition systems of differing 
technologies, besides the more conventional keyboard (typing) 
mode of input. [Ref. 2] 

Accepting the conclusions drawn in the Wright/Kariscn thesis as 
a basic premise, the Discrete Voice System (Threshola) was eliminated 
as a possible candidate for further experimentation. This decision 
was based on its record of poor performance ir. the experiment, 
and the requirement of a discrete system to speak in broken 
phrases which is unnatural and cumbersome to human beings in 
general. Since the keyboard was highly successful as an input 
medium in the previous experiment, the results were used as a 
third comparison against the two newer mediums selected in this 
experiment. Because the Wright/Hanson experiment usee the same 
NWISS wargaming scenarios, it was not only unnecessary to repeat 
the procedure, but it allowed for more concentration on the 
exploitation cf the possibilities present in the "newer media". 



The basic purpose cf this experiment was to evaluate different 
input interfaces to the IJV/ISS wargame scenarios resident on the 
Digital Equipment Corporation's VAX 11/730 computer. Having 
selected the appropriate input media (i.e., the Macintosh micro- 
computer with drop-down window technology, the VEKBEX 3000 unit 
with Automatic Speech Recognition capability and tne peripheral 
keyboard attached to the VAX computer), it was necessary to 
establish certain Measures of Effectiveness (MOE's) on which to 
base the ensuing comparisons. It was decided that the number of 
commands entered during one HWISS wargaming session would be the 
primary MCE. This coupled with the number cf Control_K's (entry 
errors made by the user) entered would provide the necessary data 
in which to answer the following questions: 1) Was any system 
more effective, and or easier to use?; 2) Was any system r.ore 
effective under the varied environmental conditions?, and 3) Was 
a clear learning curve established as a result of more practice 
with a particular input device? 

This thesis will not include a description of the construction 
process used to devise the wargaming scenarios, or provide the 
methodology used to build the operating system; since they both 
already exist. Additionally, the development of the VAX 11/780 
and the resident NWISS is already complete and will not be discusseo. 
All conclusions from the experiment and the author's recommendations 
for folio v/ - on experimentaticn/study succeeds the analysis of 
data, and will provide some insight for the future use cf this 
burgeoning technology. 
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C. SUMMARY 



This thesis comprises a description of the initial experiment , 
including all procedures and background material precedi ng its 
inception. A summary of basic speech reccg niton technology, 
macintosh microcomputer technology and a review of past experiments 
with speech recognition at KPS is provideo in Chapters II and 

III. The design process for this experiment (i.e., design of the 
scenarios, determination of environments, background of the 
subjects, and practice/training sessions) is given in Chapters 

IV, V ana VI. Analysis of the data from the experiment is provided 
in Chapter VII. All experiment conclusions and the authors' 
recommendations for further study/applications follow the analysis 
of data section. 

Appendices F and G provide detailed guidance for follow-on 
experiments using Vertex 30C0 or Macintosh microcomputer technology, 
respectively . 
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II. SPEECH TECHNOLOGY 



A. INTRODUCTION 

The use of Continuous Speech Recognition Systems has expanded 
significantly over the past three tc five years. After first 
finding success with discrete systems. ASR is now being intro- 
duced for use in critical areas of operation. The progress made 
in recent years is measured in many different ways. The meat 
obvious is the performance and capabilities of the machine. Seme 
cf the other criteria, however, are equally important. These 
would include increased vocabulary size, distinguishing between 
easily confusing words and minimizing the effects of noise and 
other interfaces. This chapter will discuss some of the theory 
behind Continuous Speech Recognition Systems. 

B. DEFINITION OF TERMS 

The real meaning of speech recognition may seer., intuitive, 
but its derivation and appropriate applications will be described 
further to preclude possible confusion, and to provide a common 
base for universal understanding. 

Speech Recognition can be defined as the process of trans- 
forming the continuous acoustic speech symbol into discrete 
representations. These symbols may be assigned proper meanings, 
and when comprehended . may be used to affect responsive behavior. 
The next major distinction is made between Connected (Continuous) 
Speech, or Isolated (Discrete) Speech. Connected Speech is that 
in which there are no significant pauses. An example would be a 



sentence spoken at a norr.al rate. It is a difficult tasw for 
computers to distinguish between where one word ends, and the 
next word begins. Isolated speech does not have this problem, 
since significant pauses (i.e., Tur n--Latne--Gn ) are often enou 0 h 
to allow distinction. The "bottom-line" goal of any computer 
system, is to rapidly anc accurately interpret the user's input, 
and then select and respond appropriately. There is no doubt 
that it is easier to make com.puters recognize words spoken in 
isolation, or at worst, spoken in broken phrases. In fact, there 
are several examples of this on the market today. By the sane 
token, however, speakirg with isolated words or broken phrases is 
difficult for humans to achieve, and hence not desirable. [Ref. A 
and 5 ] 

C. CONTINUOUS SPEECH RECOGNITION BACKGROUND 

As the cost of equipment declines, Continuous Voice System;s 
are receiving increased attention. Concentrated research started 
in 1971 when the Advanced Research Project Agency (DARPA) of the 
Department of Defense allocated $15 million dollars toward a 
program called Speech Understanding Research (SUR). It was the 
largest single project ever undertaken in speech recognition, and 
succeeded in developing machines that were capable of "understanding" 
continuously -spoken sentences involving a 1000- word vocabulary. 
[Ref. 6] Although the DARPA project was not the first to research 
continuous speech recognition, it spawned the ideas of several 
other companies who followed suit shortly thereafter. 
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The DARPA project came to an enc iri 1976 with a long-awaited 
demonstration of several new systems that could understand continuous 
spoken sentences. Even though the results were far from perfect, 
they were extremely enlightening. The Harpy System for example, 
far exceeded its design goals by correctly understanding 956 of 
the sentences spoken by five speakers. It used a 1011-word vocab- 
ulary. and a highly-constrained grammar cf sentences relevant to 
a task concerning the retrieval of documents from the computer 
memory . 

In 1973. IBM developed a system for automatic recognition of 
continuous speech that correctly recognized 96% of the 7 digit 
numbers, with a per-digit accuracy rating of better than 99%. Ir. 
1976. Texas Instruments developed a system successful in verifying 
the speaker's voice before permitting access to the medium itself. 
[Ref. 61 

At present, there is a great desire to improve the quality of 
continuous speech recognition technology. Due primarily to its 
increased speed, naturalness and mobility, researchers have 
recognized its importance in nany varied applications (i.e., Air 
Traffic Controller training, key word identification in monitored 
conversations, command and control environments, cockpit communi- 
cations. etc.), and the necessity to exploit all capabilities. 

D. TYPES OF EQUIPMENT USED 

The NV/ISS was the nucleus of this experiment. In addition, 
the VERBEX 3000 and the Macintosh microcomputer were configured 
for use as two cf the experimental input mediums, and the peripheral 
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keyboard attached to the VT1C2 terminal of the VAX 11/7 SC used as 
the third. 

1 . Continuous Speech Recognition Ecui rment 

The V E R Is F. X 3 C 0 0 is currently the "state of the art" 
regarding Continuous Speech Recognition Systems. It is designee 
for use in industrial and commercial applications, and works 
well in both high and low noise environments The system is 
capable of recognizing natural continuous speech of infinite 
length, and an output buffer of 2A0 characters per recognition 
set is its only limitation. [Ref. 7] With a maximum of four 
speech processing boards, the VERBEX 3000 can recognize up to 360 
different words spread over as many as 20 different grammars. A 
finite limit to grammar size, based on the total number of words 
and complexity of the node transition network, is necessary to 
allow the device to remain "real-time" in terms of computation 
speed and mer.ory (i.e., stored voice patterns) requirements. Thus, 
the total application may involve up to 360 words, but at any 
instant the recognizer is dealing with only one grammar subset. 
[F.ef. 8] The VERBEX 3000 system includes the Speech Application 
Development System (SPADS). This system allows the user to run 
applications/software other than those procuced by VERBEX 3000. 
Nevertheless, any program must be written in the programming 
language Pascal in order to effectively control the application. 
It is through the use of SPADS that Lombardo (an NP5 graduate 
student) was able to develop a program in which voice commands 
were used in conjuction with the wargaming scenarios used in the 
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IH.-ISS. [Ref. 8 3 Aodi tional equipment for the V EKE EX 3000 includes 
3 smell speech unit which prompts the user for the next word or 
phrase. This unit will echo the command if subsets cf phrases are 
neeaec tc complete the command. The subjects in this experiment 
used a Shure SMI 2 A headset (close-speaking) microphone ( as the 
dedicated input device), and the Siegler ADM 31 terminal to link 
the VERBEX 300 0 system with the VAX 11/780. This particular 
terminal echos all user inputs including commands and errors 
identified as Control_K's. 

2 . Laboratory Ecuicment 

This experiment required three iriput/cutput terminals 
(VT-102 series), three status board terminals (VT-100 series) and 
three Ramtek color graphic display terminals. The three ir.put/cutput 
terminals were assigned as control, umpire anc m aster. Each 
subject (as well as the umpire) had a color graphic display to 
view the scenario in progress. Status boards for the two subjects 
provided accurate ccmb3t information for the respective forces, 
and the umpire had access to all information on a special status 
board configured for that purpose. 

E. PREVIOUS NAVAL POSTGRADUATE SCHOOL STUDIES 

In the past several years, there have been a multitude cf 
experiments and theses developed at the NPS to realize the potential 
and limits of Speech Recognitor. This section will review the 
findings of a few of these studies as an attempt to summarize the 
progress to date. 
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In 1981 * research occurred which analyzed speech recognition 
as a means to improve speed and reliability in the task of imagery 
interpretation. The purpose of the experiment was to allow an 
officer to devote more time to concertrste on the object being 
identified , and less time inputir.g the information on the key oo art; . 
A discrete voice system (i.e.» Threshold. T6C0 equipment) was 
used to conduct the experiment, and the analyst found that the 
voice system was approximately 503 faster than typed data ertry. 
As far as "reporting accuracy", no significant differences were 
noted. [Ref. 9] Concurrently, in 1981 a hPS student studied the 
practicality of using voice systems in a war game environment. 
The objective here was to play the game scenario utilizing a 
discrete system (T600) and a standard keyboard. Performance 
measures that were evaluated included time required for each 
input, and the number of input errors. The game, entitled Warfare 
Environmental Simulation (WES), (later evolved into NV/ISS) provided 
the interactive, real-time wargame used in this experiment. The 
student's experiment suggested that the keyboard entry was faster 
and resulted in fewer errors. [Ref. 10] Why then was there a 
difference in outcomes between these first two studies? Apparently, 
the method of measuring the errors was different ir the WES 
experiment. The total errors for the voice entry included recog- 
nition and operator errors, while the keyboard entry totals only 
included the typing errors. For the imagery interpretation exper- 
iment, the methods used to measure errors for both input devices 
was identical. 
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Several other experiments using discrete systems were conducted 
with similar results. The "better" method depended or. the designated 
HOE. the experience level of the subjects, the extent of the 
training of the suojects, etc. 

Ir. 1984, another i'PS student developed a thesis that attempted 
to evaluate the use of continuous voice recognition technology 
in a war-game environment. The student developed the application 
software to run the MVilSS wargame using tire VERBEX 3000/continuous 
speech system. This software became the foundation for the 
Ur ight/Hanson thesis; the final thesis before tl is experiment. 
The authors of the thesis. Wright and Man sen, evaluated the 
effectiveness of three input devices (continuous speech, discrete 
speech and the standard keyboard) while running the MWISS war-game. 
The nonintuitive conclusion suggested by Wright and Manson was 
that the keyboard players did better overall. Further research 
gave seme possible plausibility for this result. During low light 
sessions almost all the keyboard subjects moved the keyboard 
panel closer to the CRT to alleviate a perceived handicap. A post 
experiment survey also revealed that the keyboard group had the 
highest typing skill level of the three groups. Additionally, 
the discrete system fared well below the other methods In the 
majority of the comparisons, and even though shorter training was 
required, incorrect recognition was more likely to occur under- 
varying levels of ambient noise and dissimilar user voice patterns. 

From the results obtained in the Wright/Hanson thesis it can 
be ascertained that while the keyboard still remains an alternative 
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for computer interface, the discrete system (however effective it 
may be) is not natural to an individual's way of speaking, and 
does not merit further investigation. Concurrently, it is also 
recognized that more evaluation of the continuous speech recognition 
system is needed, and it shall be discussed further in inter 
chapters of this thesis. 

F. PROBLEMS OF SPEECH RECOGNITION 

With any field of study there are many problems which tend to 
binder the progress of new technology; and Continuous Speech 
Technology is no different in this regard. Nevertheless, several 
additional factors must be considered when human behavior is 
involved. For instance: Can an inexperienced user operate a voice 
system? Will the system accurately recognize an individual's 
voice signals? In general, the bigger the system's vocabulary 
capacity, the more prior training will be required. Also, if a 
large number of sentences and alternative phrases need to be 
recognized, additional training may be necessary. The vocabulary 
itself plays an important role in determining the systems ability 
to recognize utterances with any degree of consistent accuracy. 
In other words, it is much easier to recognize ICO distinct 
words, than to recognize 100 "rhyming" words/phrases (i.e., a 
"light house" vs. a "lighthouse"). 

Another difficulty occurs when there are "highly constrained" 
sequences of words (i.e., stock nurabers, telephone numbers, etc.) 
or carefully formatted phrases (i.e., air traffic control commands, 
fixed-length commands with each word selected from a predetermined 
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vocabulary, etc.). These words or phrases require a acre complete 
analysis tc insure the proper context of each word . It is also 
extremely important that the speaker articulate/annunciate each 
w ore in the c o mm and. 

One of the biggest difficulties encountered when dealing with 
speech recognition is the unwanted ability of the microphone to 
pick up outside noise, or sounds from trie environment. While it 
is a fact that real life conditions may encounter additional 
noises causing degradation in the systems ability to accurately 
recognize words, the judicious use of available testing facilities 
will allow optimum conditions on a regular basis. Another possible 
solution is to use "close-talking" microphones or headsets. 
These particular devices are able to differentiate between nearby 
sounds and sounds heard at a distance. [Ref. 6] 

Cost is another very common problem with this technology. For 
Research and Development companies and commercial users, the 
initial cost is very high. At this point in time, the demand for 
continuous speech recognizers is not enough to stimulate the 
significant number of grants and contracts necessary for adequate 
research. 

The question that is uppermost in the mirds of many commercial 
investors is not whether the equipment is affordable, but is 
it necessary? Do we really need a continuous speech recognizer? 
Some people argue that "isolated word recogr izers" are adequate 
for most jobs. The fact still remains there is little doubt that 
if improved, the sheer mobility and flexibility of continuous 
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speech systems would easily surpass the cumbersome operation 
inherent in discrete systems. 
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III. MACINTOSH MICROCOMPUTER TECHNOLOGY 



A. INTRODUCTION 

Although tie ccds against its fruitier) seemed monumental in 
1979. the Macintosh microcomputer, designee by a small group of 
inexperienced en^ireers and progr ar. mers . is now recognized worlo-wide 
as a technological marvel in personal computers. Relying heavily 
on symbols and associated pictures on the screen, it is designed 
primarily to help people conquer computer phobia and be as easy 
to use as a television set. It is this ease of use attribute that 
made it so attractive as a possible testbed interface device for 
wargame simulation. [Ref. 11 ] 

This chapter will focus on some of the terminology inherent 
in microcomputers . described the origin/background surrounding 
Macintosh's innovative technology and its adaptability to experi- 
mental design, and lastly, describe the equipment that was used 
in the experiment. 

B. BACKGROUND 

The Apple Corporation ' s Macintosh microcomputer is built on 
three (3) primary ideas: 

1) second-generation Lisa technology (an expensive desktop 
computer designed for commercial use that pioneered the 
merging of gnaphics and "mouse"-oriented peripheral 
operation ) , 

2) reliability and low cost through inbred simplicity, 
and 

3) a maximumization of the synergism possible between hardware 
and software. 



Powered by the dedicated ambition cf Burrell Smith (a former 
Apple Computer Corporation maintenance technician) Steven Jobs 
(the chairman of Apple, and the Macintosh project director) and 
an elite project team of "visionary radicals", Apple has succeeded 
in creating a machine that weighs only 20 pounds, can be carried 
in a tote bag and has many of the significant features introduced 
in January of 1983 with the Lisa computer. Although it uses a 
"mouse" (a pointing device roughly the size of a pack of cigarettes) 
that permits users to give commands to the computer via arm/hand 
movements and button pushing, there are four (4) basic aifferences 
from the Lisa that make it second-generation oriented: 

1) a much higher clock speec (7.83 MHZ vs. the Lisa's 5.0 
MHZ ) , 

2) smaller memory used more efficiently through the MC68000 
processor , 

3) an elimination of add-on peripheral cards through the 
extended use of a high speed serial bus coupled with the 
implementation of "virtual slots", and 

4) it allows only one applications program to be active at a 
time (an important limitation). 

Although this latter limitation aspect seems serious in 
nature, it actually facilitates the use of multiple window operations 
and allows material to be "cut and pasted" from one document to 
another by a "clipboard" arrangement. Nevertheless, the absence 
of hardware slots and the lack of ability to run two simultaneous 
applications are the two important separation criteria from that 
of the Lisa technology. [Ref. 12] 
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By combining the functions of many stanaarc chips into eight 
programmable logic arrays, it makes it possible for the i-'.acintosh 
to keep its chip count (and therefore its price) relatively low. 
while simultaneously producing a machine that has a reliaoj iity 
factor characteristically associated with much sir; pier computers. 
With only two printed circuit boards, it was cleverly designed to 
eliminate the need for hardware fir.e-tuning during assembly or 
use. It is this combination of reliability and simplicity that 
makes it an ideal training or input interface. 

The Macintosh's hardware and software are fully optimized for 
maximum potential. This simply means that the hardware and software 
have evolved slowly ever a period of time that has provided a 
mutual "give and take" association. As a result, the video display 
hardware design greatly simplifies the software design; allowing 
it to easily draw squares and circles, scale text and graphics 
and print screen images. Utilizing this feature to its fullest. 
Tom Concannon (a NPS graduate student) was able to develop the 
MV.'ISS wargame interface program in the BASIC language to allow 
the experiment subjects to play the various wargame scenarios. 
This program activated the "user toolbox", which is primarily 
responsible for manipulating all the various relative relationships 
between different units/accessories in the MC68000 processor. By 
stimulating the window and menu managers, the subjects were then 
able to use the mechanical mouse to select wargame corr.nancis from 
a two-dimensional matrix of menu items. The selected input was 
coordinated through a resource manager facility which perpetuated 
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the eventual interface necessary for communication and comii.ard 
actuation on the VAX 1 1/730 computer. An example of each menu 
column followed by all the possible user selections is contained 
i r: the pictorial section of the Macintosh Kic recompute r User's 
Guide (see Appendix G). [Ref. 3] 

C. EXPERIMENT HARDWARE DESCRIPTION 

The Macintosh has received numerous accolades for its impressive 
design. It is truly a highly developed microcomputer that effectively 
uses a mechanical mouse peripheral and a 32 bit microprocessor ir. 
a lightweight portable package. It is its portability, functional 
simplicity and ease of use that make it the ideal selection for a 
user interface device. 

The Macintosh microcomputer is a self-contained unit (9.75 by 
9.75 by 13.5 inches) consisting of a main unit, keyboard and 
mechanical mouse. It uses standard AC external power, has two 
serial ports and an integral Sony 3.5 inch disk drive which holds 
up to 400 Kbytes of information. Its standard 128 Kbytes of RAM, 
and 64 Kbytes of ROM were reconfigured before the actual experiment 
to 512 Kbytes to accomodate the slight command entry delay experi- 
enced by users at the lower configuration setting. The nine (9) 
inch monochrome monitor provided an adequate display of all 
pictorials, and its 58 key detachable keyboard served as a comfor- 
table medium for a backup typing input mode. The main source of 
user input was a small mechanical mouse which is physically moved 
in a rectangular series of hand movements to position an onscreen 
arrow to the desired menu. Keeping the index finger positioned on 
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a conveniently mounted button, the user manipulated the mouse to 
position the arrow, and used the button to extenc the desired 
menu and select the appropriate command for subsequent implemen- 
tation. A detailed description of the mechanics involved in user 
operation is again available by referencing Appendix G. [Refs. 12 
and 2 ] 

RAI-iTEK high resolution color graphics monitors were used to 
display the game scenarios to the subjects at their respective 
positions, and the digital VT100 terminal was used as a status 
board for each NV.'ISS session. The configuration of the laboratory 
itself will be further described in Chapter VI, and a pictorial 
display of the physical layout provided (see Figure 6.1). 

A conventional cassette tape recorder was used to play a 
pre-recorded background noise tape during each t.'VJISS session. One 
eight inch, eight ohm speaker was placed at a pre-established 
distance from each subject, and an audiometer was used to verify 
the decibel settings for each light/noise environment specified. 
The pre-recorded noise consisted of a gathering of approximately 
thirty (30) people speaking in a conversational tone within an 
auditorium. [Ref. 2] 

D. SUMMARY 

This chapter has served to discuss some of the termirology 
inherent in the design/description of the Macintosh microcomputer, 
and has presented some of the background theory and history 
behind its origin. Although there were no previous wargame scenario 
experiments conducted at the NPS involving the use of the Macintosh 
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(or similar technology) as an input medium, its ease of use. 
its overall potential prompted its employment in this experim 
Finally, a description of the actual microcomputer hardware 
associated laboratory equipment included in the experiment 
provided . 
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IV. THE NAVAL WARFARE INTERACTIVE SIMULATION SYSTEM (NV/ISS) 



A. INTRODUCTION 

This chapter contains a brief Gescription of NWISS arc: its 
support role in this experiment. All scenarios developeo for the 
experiment, the measures of effectiveness selectee ana the methods 
usee to collect data and compare the two command input technologies 
are also described. 

B. WHY NWISS WAS SELECTED 

NWISS is a real-time, user-interactive simulation of r.aval 
warfare. Its mission was originally to train senior Naval Officers 
ir force-level tactical decision making and management of command 
and control. At the Naval Postgraduate School, NWISS resides on 
a VAX 11/780 computer, and the peripheral VT1C0/102 terminals and 
PAKTEK graphics terminals provide the necessary input/output 
modules when grouped together. Voice equipment (i.e.» the VERTEX 
3000), anG "drop-down" window microcomputer technology (i.e.» the 
Macintosh microcomputer) were used as interface devices in tne 
available modules, and all inputs and outputs were accomplishec 
by traired subjects. NWISS was loaded, controlled and monitored 
from a third "commanG" module during these experiments. The 
purpose of NWISS in this experiment was to stimulate a stressful 
environment that, would allow user intervention via NWISS commands. 
From the user's point-of-view, it has a large vocabulary of 
commands to change the view of the tactical situation in the 
battle arena, cause changes in the course and speed of vehicles 
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in one force, and launch air assets with user defines weapon 
loads and rrissions. UV/ISS can also show the status of the user's 
forces on several menu selected displays. For this experiment, 
each station consisted of a command input terminal, a force 
status terminal and a videc monitor to graphically display force- 
disposition and identity. [Ref. 2] 

NWISS was used to support this experiment because it: 

1. Offered the ability to present identical tactical situations 
to two isolated subjects simultaneously . This controlled 
environment allowed a side-by-side comparison of the command 
input methods chosen by the isolatea subjects when confronted 
by identical tactical stimuli. Additionally, the system 
was readily available at the MPS, and a pool of trained 
operators 'were accessible for "on-line" consultation. 

2. Provides a large system vocabulary. Consensus of opinion 
was that if a small vocabulary was chosen to run the 
experin.ent, the subjects may become bored through sneer 
command repetition. Approximately 150 words/utterances were 
deemed necessary to properly conduct the variety of 
inherent missions within the capabilities of an Aircraft 
Carrier Eattle Group. Twenty commands relating cirectly 
to the control of the tactical force display were also 
included. Tables of the complete vocabularies for the 
two input technologies are included in Appendices F a G. 
These vocabularies were predetermined and were limited by 
the size of the memory of the VERBEX 3000 system. Using 
the forces and commands in these subsets of the NwISS 
dictionary, four previously developed scenarios were used 
as a medium for stimulating subject interest, and for 
generating a meaningful MOE. [Ref. 14] 

In all four scenarios, the Orarge and Neutral force actions, 
defensive posture and radar emissions control (EMCOIJ) status were 
preprogrammed . Orange aircraft were eliminated from this experiment, 
to avoid unnecessary system program complexity. The two available 
views of the world for the Blue forces were simulated on approxi- 
mately the same latitude, but were physically separated in longitude 
by 500 nautical miles at mid-ocean. 
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liV.ISS is limited to a maximum, of six sets of forces, and the 
Grange and Neutral forces were each structured appropriately to 
reflect one view. The specific process for assembling these 
various views and overlapping them is explained in some detail in 



a previous NFS thesis by Owens and Drown. Cf'ef. 15] Basically, 
there were two separate views for the Blue forces which allcwec 
the subjects to individually control their own ships and planes. 



C. SCENARIOS 

Four unique scenarios were used in order to effectively place- 
subjects in situations requiring the use of the available t.’UISS 
vocabulary, and to stimulate individual/different responses. This 
section describes the basic opening force positions for each of 
the scenarios and the accompanying military situation briefing 
for that scenario. The actual situation briefings given to the 
subjects are included in Appendix A. 

1 . Scenario A 

One aircraft carrier, twelve P3C aircraft, two frigates, 
and an attack submarine are assigned to the Blue forces. Orange 
forces include one aircraft carrier and six surface combatants, 
and are escorting eight neutral merchant ships in a convoy that 
is moving south as shown in Figure 4.1. The merchant ships are 
reportedly carrying nuclear and chemical weapons to an unidentified 
Orange ally. Hostilities have been declared between the Blue 
and the Orange forces and the objective is to identify and destroy 
the Orange escort vessels without involving/destroying the merchant 
ships. 
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Scenario B 



p 

Blue forces consist of two P3C aircraft to provide tre 
necessary targeting information. one aircraft carrier anc two 
flights of five attack aircraft. Four Grange surface combatants 
are positioned between these Blue forces, anc twenty r: ere rat; t 
vessels are at anchor outside a fictitious harbor facility waiting 
to unload their cargo. Hostilities have not been declared between 
Blue and Orange forces, although tensions remain high. The Blue 
force’s mission is to use the assigned airborne aircraft, and the 
remaining aircraft carrier based assets to conduct a strike on 
the merchant vessels. Rules of engagement do not permit firing by 
the Blue forces until the Orange forces have fired on them. From 
the opening positions as shown in Figure 4.2. neither the merchant 
ships nor the Orange combatants are visible to the Blue forces, 
but a northerly course is dictated by the mission. 

3 . Scenario C 

A favorable environment is present in which Blue tasking 
is to locate and identify as many objects as possible and designate 
them as Friendly. Neutral cr Enemy. Blue forces have one P3C anc* 
four F14A aircraft to accomplish this mission, and Combatants 
for both Elue and Orange forces, as well as several merchant 
ships and fishing vessels, are all in a 400 X 500 nautical mile 
rectangle (as depicted in Fig. 4.3). No control of the Blue 
surface forces was permitted to complicate the tasks of location 
and identification. 
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4 . S o e t i a r i c D 

Two carrier cattle groups of approximately equal strength 
are pitted against each other. Blue forces are specifically 
tasked to locate and identify the elements of the Orange force 
battle group, hostilities have not been declared, but the subjects 
were allowed wide latitude in their response to any Grange force 
aggressive action. The opening positions in Figure 4.4 place the 
Grange force carrier battle group to the North cf the Blue force 
group; with the Orange force headed North and tne Blue force 
headed to the Northwest. 

D. MEASURES OF EFFECTIVENESS 

All subjects were familiar with the NV/ISS war garni r g scenarios. 
Some of the commands (i.e., changing the scale of the display) 
were easy to accomplish and rapidly assimilated, while others 
(such as the launch sequence) required patience and consider a ole 
practice to implement correctly. When the subjects received their 
mission briefing before each session, they were told some of the 
measures of effectiveness by which their success or failure would 
be determined. Included were such things as the number of merchant 
vessels sunk or damaged in Scenario B, the number of ships correctly 
located and identified in Scenario C, and the number of Blue 
forces lost to hostile fire ir. Scenario A (sec Appendix A). In 
each case, the subject was urged to accomplish the goal wit) in 
the allotted time period (i.e., 40 minutes). These liOE's were 
false sets of MOE’s, ana although not used for analysis, were 
intended to stimulate the subjects to achieve a meaningful goal. 
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LEGEND: © BLUE SURFACE COMBATANT 
BLUE SUBMARINE 
© ORANGE SURFACE COMBATANT 
n BLUE AIRCRAFT 
□ NEUTRAL MERCHANT SHIP 




Figure 4.1 Scenario A 
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LEGEND: P BLUE AIRCRAFT 

BLUE surface combatant 
O neutral merchant ship 

□ ORANGE SURFACE COMBATANT 
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Figure 4.2 Scenario B 
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Figure 4.3 Scenario C 
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Figure 4.4 Scenario D 
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T'ne actual measures of effectiveness are related to trie 
number of Hl.ISS commands entered ana their ccrresponai ng accuracy. 
The number of commands entered was considered to be the rrcst 
precise indicator of subject mastery over the input technique. It 
was felt that the four scenarios, with the applicable combinations 
of light and noise levels, provided the opportunity for the 
subject to use commands available under varying environmental 
conditions. Real-world time constraints and accompanying pressures 
were impressed on the subjects by the schedule, and by the delib- 
erate. overt actions of the facilitators. Since all the sessions 
lasted the same number cf game n ir.utes. the number of commands 
entered was essentially a rate of entry. The astute subject could 
have increased this rate significantly simply by repeatedly 
varying t..e display radius. By stressing the fictitiously generated 
BCE's, this type cf performance was indirectly discouraged. 

It was determined through supervisor consultation and group 
advisory discussion that accuracy can be measured to some extent 
by the number of times the "Control_K" command is used to delete 
an attempted command. Nevertheless, there are some limitations 
to using " Control_K" as a measure of the number of errors made by 
subjects. Since it aborts the current command and does not execute 
it. tne "Contrcl_K" could be viewed as a change of mind on the 
subject's part. From prior experience with the NWISS game it is 
as easy tc enter a completed command and issue a subsequent 
correction, as it is to abort the command completely. [Ref. 2] 
There are also two equipment related factors that detract from 
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the use of "Ccr.trol_X" as a r, error counter. First, in the VEREEX 
3000 speech recognition system, after the keyboard has been used 
to enter an K'.'ISS command not included ir, the 15b word vocabulary, 
the "Cor.trcl_K" commanc is necessary to return to the beginning 
of the con, r. and input sequence. Therefore, this use of "Controi_k" 
does not indicate an error, out the correct use of the system. 
Secondly, it could be that the subject's uttenance/word was not 
recognized by the system anc the subject elected to enter the 
command manually. The result is the same as in the first case: 
the abort command must be used to return to the start of a new 
command input sequence. In a deliberate attempt to alleviate this 
particular "discrepancy", the authors' evaluated the printout of 
the sessions immediately following their execution so that memory 
of the subject's rationale behind the selection of a Cor.trcl_K 
input could be interjected, and subsequently discarded if it was 
not intended as an error correction. Overall, despite these 
considerations, it was felt that the "Contrcl_K" command provided 
the best measure available of errors made in entering commands; 
although it still may not be as stror.g a measure as the number of 
commands entered. 

E. DATA COLLECTION 

Data collection was accomplished through several utility 
programs of the VAX 11/780. All commands entered by the two 
subject's command terminals were sent to separate files in memory, 
as was the record of the umpire's game control terminal. This 
process provided a record of all subject activity, ano kept a 
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historical account of all engagements, weapon firings and resultant 
damage displayed on the umpire's terminal for further study. The 
output was then narid-cour.ted to determine the number of comr arc's 
entered and the number of errors committed. 

One additional comment is warranted regarding how the number 
of commands entered was actually determined. The aircraft launch 
sequence is widely used, and is the most lengthy arid difficult 
command. Since the aircraft launch sequence involves at least 
five lines to be complete, and could potentially be aborted 
anywhere within those lines, each line of the five that was 
correctly entered was counted as a command entered. 
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V. EXPERIMENT BACKGROUND 



A. INTRODUCTION 

Several independent factors were consicereo crucial to tue 
success of this experiment. First vjas the cor. tinueo operational 
status of trie critical "hardware" (i.e.. the VAX computer. the 
Nl/ISS wargar.ie. the VERBEX 3000 system and the Macintosh computer). 
Secondly, a total of 24 subjects were needed to participate in 
the experiment. Decisions such as the length of each session., the 
randomness of the sequences and the combinations of scenarios and 
environments were also of major importance. This chapter will 
describe the actual training and practice sessions using the 
VERBEX 3000/ Macintosh microcomputer technology, discuss the 
background of the subjects, and elaborate on the details surrounding 
the design of the experiment. 

B. BACKGROUND OF SUBJECTS 

The 24 subjects who participated in the experiment hac a 
myriad of backgrounds. Twenty-three of them were armed forces 
officers (i.e.. U.S. Navy. U.S. Army. U.S. Air Force, etc.), and 
all were studying Command. Control and Communications (C3) at the 
Naval Postgraduate School (NPS). Nineteen were male and four were 
female. The twenty-fourth subject was a male civilian computer 
operator employed at NPS. 

All the students participated in the experiment in conjunction 
with OS 4602. a required Operations Research course titled Command. 
Control and Communications Systems Evaluation. A previous Operations 
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Research course, OS 3603 (Simulation and Wargaming) , introduced 
the students to the NWISS background and symbology usee for the 
experiment, and a similar course, OS 3404 (Man-Machine Interaction) , 
introduced them to Discrete Automatic Voice Recognition Theory. 
All the subjects were familiar with the basic facilities; that 
is, the digital VT100/102 keyboard terminals and the Ramtek 
graphics monitors. None of the subjects had ever used a continuous 
automatic speech recognition system, and some had never used the 
particular headset-type microphone incorporated by the VERBEX 
3000 unit. Additionally, out of all the Macintosh players, only 
five had ever used a mouse, and due to the specific nature of the 
software program, it woulc be the first time anyone had played the 
NWISS game using that device. The civilian programmer w as to be a 
VERBEX 3000 player, and although he was familiar with the NWISS 
game, he had never played it using the continuous voice interface. 

The 24 subjects were divided into two equal teams of 12 
people. Each team would only use one input device (either the 
VERBEX 3000 or the Macintosh). Selection was mace randomly, and 
on that basis a VERBEX 3000 player and a Macintosh player were 
partnered so they could be scheduled for each session simultane- 
ously. Similar classroom schedules were the only matching prereq- 
uisite, and although the amount of exposure to the NWISS was 
equal for all subjects, a similar statement could not be made 
concerning each individuals competence v/ith the same. 
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C. ENVIRONMENTAL CONDITIONS 



The experiment was constructed to simulate a military command 
and control center. To accurately represent this, four aifferent 
environmental conditions were used. One of four combinations of 
noise and licht levels was established for each run of the exper- 
iment, and one of four different scenarios was used to allow each 
player to experience a different set of circumstances and environ- 
mental conditions for each individual's wargaming sessions. 
Additionally, by alternating scenarios it helped to preclude 
familiarity, and any possible unfair advantage achieved through 
intimate knowledge of one scenario in particular. 

Noise and light conditions used for the experiment are summarily 
described as follows: low noise (65 ab) , high noise (85 db) , 
normal lighting (approximately 1 f oot-lambert ) and lov; lighting 
(lights completely out; approximately .05 f oot-lambert) . The 
introduced noise was measured in decibels with an audiometer 
(on the C-scale) , and readings v/ere taken at the subject's "ear- 
level" with the stereo speaker approximately 2-3 feet away. 
Light conditions were measured with a light meter. Normal light 
conditions of 1 foot-lambert were achieved by maintaining the 
overhead track lights on and directed at a 45 degree angle toward 
the nearest wall. With the lights out in the lev; lighting setting 
the only lighting available to the subject was the light emitted 
from the associated CRT and Rarctek graphics screen. 

To the greatest extent practicable, physical partitions were 
used to maintain the appropriate noise/light conditions throughout 
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each session. Full cooperation was experienced from the other 
users of the C3 war gaming laboratory, and a stable , controlled 
experimental atmosphere maintained (see Figure 6.1, Chapter VI, 
for a complete Laboratory Layout) . 

D. SCHEDULING OF SESSIONS 

The NWISS experiment took place in the C3 v/arcaming laboratory 
at the Naval Postgraduate School. It was intended to be run in 
lieu of the subjects scheduled laboratory period for their OS 
3603 course. The subjects v/ould participate individually in four 
sessions; each run was scheduled to last for 40 minutes. A short 
br ief/debr ief period v/ould make the entire session approximately 
50 minutes in total length. 

The experiment was conducted from 15 October 85 through 28 
October 85, and during that period a total of 96 sessions v/ere 
recorded. The C3 laboratory proved to be an ideal location for 
the experiment. Because the NWISS game is resident on the VAX 
11/780, and the VAX, the VERBEX 3000 and the Macintosh computers 
v/ere all located in the same laboratory, they v/ere all easily 
configured and able to communicate v/ell with each other. 

The experiment was conducted early in the Fall academic 
quarter to alleviate traffic congestion throughout the laboratory 
by non-participants. Since the laboratory was divided into three 
equal bays, each player was completely separated from the other 
player, and in almost total isolation from other laboratory users. 

Actual scheduling w as done on a random basis, and changes 
v/ere only made if conflicts arose betv/een team members. The 
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cooperation exhibited by the subjects in this regard was greatly 
appreciated as only a few last minute changes hac to be made. See 
Appendix E for a complete listing of subjects, and the actual 
experiment schedule of sessions. [Ref. 16] 

E. SCENARIO/ENVIRONMENT SEQUENCE 

A separate scenario and environmental condition was assigned 
for each session, and each team played the NWISS under all four 
combinations. For example. Team 4 played the NWISS using Scenario 
A and Condition 4, Scenario B and Condition 3, Scenario C and 
Condition 2, and finally Scenario D and Condition 1. The order of 
assigning scenarios was counterbalanced for sequential positioning, 
and for both preceding and following treatment effects. [Ref. 
17] (See KEY in Appendix E.) 

The assigning of environmental conditions was also adjusted 
using the same methodology. The schedule was adhered to as closely 
as possible, and even with three separate last minute session 
changes, all subjects played each scenario and environmental 
condition once. 

F. CONSTRAINTS 

Due to its secure environment, and the fact that all the 
wargaming scenarios were already available on the VAX computer, 
the C3 laboratory was the perfect place for running military 
wargames. Nevertheless, certain features inherent to the laboratory 
itself led to the following constraints during the experiment: 

1. NWISS is the only wargame available at the NPS that has 
the software available to allow the use of the VEREEX 
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3000 equipment for the input of voice commands. The 
experiment was therefore constrained to usinq the VERBEX 
3000 system for the continuous voice input. 

2. The physical layout of the laboratory itself restricted 
the placement of the VERBEX 3000 and Macintosh input 
devices to keep a lighted section of the laboratory open 
for other student use. 

3. The versatility of the laboratory allows it to be usee 
for a myriad of student/instructor projects. Because of 
this fact the experiment had to be conducted at a time 
when the laboratory could be exclusively reservea, to 
prevent interference with the established environmental 
conditions. As a result, it was necessary to complete 
the experiment in October. Additionally, the sessions 
had to be designed to coincide with the regularly scheculed 
laboratory periods of the OS 4602 students in order not 
to conflict with their individual schedules. The combination 
of all these particular constraints prudently limited the 
actual sessions to four per subject. Each session lasted 
approximately 50 minutes to allow sufficient time to 
properly brief the subjects and collect pertinent data. 

4. The subjects were randomly selected, and permanently 
assigned to only one mode of command input due to the 
previously mentioned time constraints, as well as to 
increase their personal proficiency with the assigned 
input device. 



G. TRAINING 

l. Gene ral 

As delineated earlier, all students were assigned to 
either the continuous voice recognition system (VERBEX 3000) or 
the Macintosh microcomputer located in the C3 Wargaming Laboratory. 
Each individual was given a thorough briefing on how to use the 
appropriate device, and ho w to train the equipment to either 
recognize their unique utterances of the designated NWISS commands 
(for the voice) , or enter the necessary commands selected from 
the appropriate "drop-down" display window (for the Macintosh) . 
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2 . Environment Purine Training Sessions 



Normal laboratory lighting conditions were allowed through- 
out the training- sessions. This normal level of one tc two fcct- 
lamberts would be employed during two of the experiment's four 
simulated environmental conditions, and simultaneously facilitated 
NWISS phraseology reference for the user's ref amiliar ization/ 
retention . 

A cassette tape supplied continuous background noise at 
a level of sixty-five decibels. A previous voice experiment had 
determined a direct correlation between the consistent use of 
this decibel noise setting and the best possible climate for 
voice recognition. [Ref. 18] Besides its acceptance as the best 
noise level for recognition accuracy, it was discovered that the 
background noise with the presence of a fixed air vent recirculation 
fan would provide the minimum level of noise present throughout 
the course of the experiment. 

3. VERBEX 300C Voice Training 

To allow the VERBEX 3000 system to react properly to the 
150 preprogrammed NWISS command words, it was necessary for all 
designated voice subjects to engage in two specific phases of 
voice recognition conditioning. The first phase required the 
consistent repetitious uttering of all 150 words individually in 
order to allow the VERBEX 3000 unit to logically associate each 
word with each users unique voice characteristics, and then 
physically enroll this "transformed" version of the word into its 
memory for later recall/recognition. This procedure took an 
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average of 40 minutes for each user, and although a standardized 
pronounciation of the word was supplied and encourageG , the 
subjects were free to substitute any utterance they desired for 
the VERBEX 3000 prompt (if it was remembered and used during the 
actual wargarcing sessions). In the second phase, a maximum of 
three of the 150 possible utterances were concatenated, and then 
grouped by similar usage into nine specific grammars or sections. 
Each partial command phrase v/as then repeated continuously, and 
as consistently as possible (using voice inflection and volume 
control simultaneously ) by the user. By design, each utterance 
v/as repeated approximately 20-23 times in different sets within a 
specific grammar. It was also recognized that although some of 
the utterances v/ere unique to a specific grammar, commands such 
as "Control^” were common to all grammars, and therefore trained 
more than the rest. This phase was the most demanding anc: repetitive 
in nature, and required approximately 5.5 hours to complete. 
(Ref. 19] 

At the conclusion of the enrollment and training phases 
each individual received a 30 minute practice session while 
interfacing v/ith an KWISS wargaming sample scenario. This was a 
monitored exercise stressing verbal consistency, volume control 
and refamiliarization v/ith NWISS commands. For a complete listing 
of all the grammars, the individual procedures involved with 
introducing personnel to the environment and all of the individual 
words that v/ere enrolled, refer to the MvISS/VERBEX 3000 Users 
Guide, Appendix F. 
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4 . Macintosh Microcomputer Training 

While cor.n-.and input accuracy in the vcice environment is 
based on repetition coupled with consistency, the point select 
method inherent in Macintosh's mouse requires manual dexterity 
and practiced digit manipulation. To allow the Macintosh user to 
interface with the wargaming scenario resident on the VAX computer, 
a program was written in the language BASIC to provide the user 
with command words from a limited vocabulary. The program simulates 
the VT100 terminal in that it allows information communicated to 
the VAX to be displayed on the screen for analysis. It permits 
the sequential selection of words ana limited phrases through an 
appropriate column header and its associated "drop-down" window; 
all operated and selected via mouse technology. A capability also 
exists for the user to enter the manual mode and type commands to 
the computer using the attached keyboard. 

Training for the Macintosh was accomplished in an "offline" 
mode which allowed the user to make full use of all operational 
commands, without the response of the VAX computer. Each user was 
given a 15 minute orientation brief on the use of the mouse, and 
the subsequent self-generated training in command selection 
averaged 35-50 minutes. 

H. PRACTICE SESSIONS 

1. Initial Individual Tasks 

After training their individual speech patterns the 
subjects were given written instructions on how to bring up the 
NWISS game and initialize the VEEBEX 3000 and Macintosh 
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microcomputer . They were given a list of approximately 50 separate 
KWISS tasks to perform by issuing commands using voice and the mouse 
peripheral input. While the lists v/ere different for the continuous 
and Macintosh inputs because force names and call signs were 
different, the general set of tasks v/ere the same. The tasks v/ere 
very specific in nature to require the subjects to put them into 
acceptable NWISS format, and then enter the commands as a recog- 
nizable speech or Macintosh input. 

There v/ere several purposes for the individual practice 
sessions described above; especially for the VERBEX 3000 users. 
It helped to identify problems with recognition (so that subjects 
could retrain vocabulary/refresh their memories), and it was 
useful in teaching subjects to take a general task and convert it 
into KWISS format. This w as especially important for VERBEX 3000 
users in that the necessary pauses between phrases v/ere required 
for the different grammars. Most users initially thought they 
were having recognition problems, when in fact they were net 
pausing in the proper places. The authors participated in these 
practice sessions (one on the VERBEX 3000 and one on the Macintosh) 
so that a better feel for the problems encountered by the subjects 
could be experienced first hand. (Ref. 2] 

Enhancement of the experimental design v/as achieved by using 
subject training that allowed the identification of "obscure" 
NWISS vocabulary not often trained by the VERBEX 3000 users; or 
inadvertently overlooked by the Macintosh subjects. There was an 
additional problem with VERBEX 3000 because while an individual 
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word may have been trained, it may not have been included ir: all 
grammars or phrases. These imposed limitations were taken into 
account at the scenario design phase by Wright and Hanson in 
their thesis, and since the same scenarios were used ir. this 
thesis they were also fully i ncor porated . 

2 . NWISS Practice Sessions 

All subjects went through a practice NUISS session before 
the actual experimental testing, and (as described in Chapter VI) 
all subjects were assigned to either continuous voice. Macirtosh 
microcomputer or keyboard entry. There were several reasons for 
this practice session. Since all of subjects had recently completed 
a separate classroom experiment using MWISS, it was necessary to 
refamiliarize the voice users with the names of the forces for 
which they had previously trained their voice patterns. In addition, 
it was found that in the majority of cases (due to the large 
number of students present in each group), all the subjects were 
not able to rotate through the specific position that allowed 
them to actually input commands during the running of the session. 
This sample scenario gave each individual an inkling of the type 
of commands to be performed during the actual experiment. All 
subjects had access to status boards shoving their available 
forces, which made the remembering of names less of problem with 
continued practice. 

The practice sessions proved to be very instructive for the 
authors during the actual preparation for the experiment. These 
sample sessions used the actual test experiment format, and the 



52 



observance of all the groups in a "dry run" insured that procedures 
used later would be acceptable. It also served to verify that 
there was ample time to complete the session and gather resultant 
data in the scheduled 50 minute periods. All environmental conditions 
were examined, and the correct positioning of stereo speakers anc: 
settings on rheostats was noted to insure equal noise and light 
at both positions. This also served to insure that the settings 
could be easily duplicated during the actual sessions. 

3. Conduct of the Practice Session 

The authors had prepared the laboratory before the subjects 
arrival so that all equipment was turned on, the appropriate 
NWISS scenario was initialized and placed in pause mode and the 
voice patterns for the intended VERBEX 3000 user v/ere loaded into 
the system. A command to save the session was included for post 
analysis to insure that the desired data w as actually being 
collected. 

Before the start of the practice session, the subjects v/ere 
given an administrative briefing about the practice run, and 
concerning specific instructions for the test sessions. The exact 
items covered are shov/n in Appendix C. Following the administrative 
briefing, the subjects were briefed on the particular practice 
scenario. Each individual was supplied with a sample list of 
KWISS commands to be used for reference during the session, if 
needed. A verbal briefing on the mission was also given to all 
subjects, and any questions were answered. 
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The KWISS game was allov/ed to run for approximately 30 
minutes. At the completion, the subjects departed and, if another 
croup was to come in, the authors prepared the laboratory accord- 
ingly. Ada i t ionally , copies of the photographed game screens 
were printed at this time. As was previously mentioned this 
procedure v/as only performed for the first few sessions to verify 
correct information, and to determine how long the entire analysis 
process would take. Printouts were not necessary for all the 
practice sessions because the data v/ould not be used in the experiment. 
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VI. CONDUCT OF THE EXPERIMENT 



A. INTRODUCTION 

This chapter will describe the layout and control of the 
laboratory, the actual conduct of the experiment and the method 
used to record the individual sessions. 

B. LABORATORY ENVIRONMENT 

1. Configuration 

The NWISS Command and Control laboratory was divided into 
two independent bays (each configured approximately sixteen 
feet high by eighteen feet in length) and w as used to simulate 
the environment for the experiment. The following equipment for 
each of the two input modes was selectively located in its respective 
section : 

1) a Ramtek graphics screen with a separate stereo speaker 
beside it for producing background noise, 

2) a VT100 game status terminal which was used tc retrieve 
game information only, and 

3) a player terminal (VT102 for the Macintosh or ADM-31 for 
the voice) which allowed commands to be physically entered 
into the computer. 

Using the ADM-31 as a validation device for character strings 
enroute to the computer, the voice subjects v/ere able to monitor 
their inputs, change/cancel incorrect entries and use the keyboard 
(on isolated occasions) in the event voice recognition proved to 
be an insurmountable problem. The laboratory's physical layout 
provided the subjects with easy access to all terminals, and the 
experiment proctors could evaluate the progress of each session 
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using the VAX/NWISS control station in the Control Bay. Additional 
equipment was located in each bay, but was separatee by a partition 

net oirectlv associated w i t h the experiment itself (see 
ricure 6.1 for a complete Laboratory Layout). 

2 . Laboratory Control 

Personnel traffic w as not restricted during the individual 
vc ice/Mac intesh practice sessions, but during the actual test 
sessions, access to the laboratory was more restrictive in nature. 
While the experiment was in progress all personnel v/ere requested 
to use the rear entrance to preclude the distraction of the 
pneumatic sliding door, and to prevent uncesirable/uncontrolied 
light from entering the controlled surroundings. Laboratory users 
not involved in the experiment v/ere advised that use of the 
facility v/as defaulted to whatever noise and light environment 
was currently in use during the experiment. Due primarily to this 
precautionary measure, the environmental conditions which had 
been adjusted and measured before each run v/ere adequately 
preserved . 

C. TEST SESSIONS 

The procedure for the actual test sessions was an exact dupli- 
cation of the practice sessions. As before, prior to the arrival 
of the subjects, the proper NWISS scenario v/as initialized and 
the vcice patterns for the subjects scheduled v/ere loaded into 
the machines (see Appendix B) . The subjects were briefed in 
accordance with Appendix C and provided with a copy of the applicable 
scenario for their perusal. Any peculiarities within a given 
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SPEAKER 

RAMTEK GRAPHICS MONITOR 
PLAYER TERMINAL (ADM31 OR VTI 02) 
MACINTOSH PLAYER TERMINAL 

STATUS TERMINAL 
GAME TERMINAL 



V 



A 



VERBEX 3000 UNIT 
AMPLIFIER AND TAPE RECORDER 



Figure 6.1 Laboratory Configuration 
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scenario which the subjects woulo not normally have experienced 
previously were pointeo out, and any questions were answerec:. 

Following the briefing, the actual NWISS session began. After 
the subjects went to their positions, the proper noise and lighting 
levels were set, and the game was started. All the sessions were 
run for forty minutes. Personnel who had repeated difficulties 
we re given assistance as necessary. A^t the conclusion of each 
run, copies of the garae sessions were printed out and annotated 
for later classification. The forty minute garae time point was 
noted on the printouts, and the number of commands and "Control_K ' s " 
entered v/ere computed. [Ref. 2] 

While the majority of the experiment sessions were completed 
as planned, some rescheduling was needed due to the absence of 
personnel, or to some unexpected NWISS game problems. Rescheduling 
of personnel created no particular problems because each input 
method was used to play an individual NWISS session, and the 
proper scenario and environmental conditions could be reset for 
any individual makeup as required. 
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VII 



DATA ANALYSIS 



A. INTRODUCTION 

The purpose of this experiment was to evaluate the effectiveness 
of two computer input devices under various environmental concitions. 
Since a v/argame scenario was used, the conditions simulated 
tactical or Command Information Center (CIC) environments. The 
hypothesis for the experiment was there woulc be no significant 
difference between using the VERBEX 3000, the Macintosh or the 
keyboard systems. As stated earlier, the results were based on 
the positive Measure of Effectiveness of comparing the total 
number of commands entered, and the total number of mistakes made 
while entering the commands. 



B. DATA SUMMARY 

The number of commands entered was determined by summing the 
"order entered" lines. As previously noted, the only exception 
was the aircraft launch sequence v/hich was counted as either four 
or five; depending on whether or not "weapons were loacec". The 
mistakes were determined by summing the applicable Control_K's 
that were correctly and deliberately entered. Raw data for the 
experiment is listed in Appendix E. 

Table 7.1 is the key which describes the environmental conaitions 
used in the experiment. It should be used as a reference when 
analyzing the other accompanying tables and figures in this chapter. 
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TABLE 7.1 



ENVIRONMENTAL CGND ITIONS 

NOISE: 

LOW - 65 Decibels (db) 

HIGH - 85 Decibels 

LIGHT: 

NORMAL - 1 Foot-lanibert 
LOW - .05 Foot-lambert 

Table 7.2 sunmatizes the mean number of commands entered, 
based on the variable ncise/light conditions. Figure 7.1 oisplays 
the data in graphic form. 

TABLE 7 . 2 



MEAN NUMBER OF COMMANDS ENTERED 
Environmental Condition: Noise/Light 



Input 

Device 


Low 

Norn; 


Sta 

Dev 


LOW 

Low 


Std 

Dev 


High 

Norm 


Sta 

Dev 


High 

Low 


Stc 

Dev 


Avg 


Verbex 


55.3 


23.4 


69.8 


23.4 


40.8 


15.6 


60.7 


27.3 


56.7 


Macintosh 


28.2 


9.4 


43.4 


10.0 


44.4 


16.8 


47.2 


11.8 


40.8 


Keyboard 


74.1 


15.9 


71.7 


19.6 


79.0 


27.8 


71.5 


19.5 


74.1 
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MEAN COMMANDS ENTERED 



L = Low; H = High; N = Normal 





KEYBOARD 



Figure 7.1 Mean Commands Entered vs. 

Environmental Condi t ions 
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Table 7.3 summarizes the Mean Number of Control_K's entered, 
based on the variable noise/light conditions. Figure 7.2 displays 
the data in graphic form. 



TABLE 7 . 3 

MEAN NUMBER OF CONTROL_K ' S ENTERED 
Environmental Condition: Noise/Licht 



Input 

Device 


Low 

Norm 


Std 

Dev 


Low 

Low 


Std 

Dev 


High 

Norm 


Std 

Dev 


High 

Low 


Std 

Dev 


Avq 


Verbex 


8.3 


8.7 


4.5 


2.3 


5.3 


3.8 


5.3 


4.1 


5.9 


Macintosh 


7.8 


3.7 


5.8 


2.4 


6.2 


3.5 


7.8 


4.2 


6.9 


Keyboard 


5.6 


4.3 


7.7 


8.0 


7.6 


3.6 


7.3 


6.4 


7.0 


Table " 


'.4 summarizes the 


Mean Number of Commands entered 


based 


on the trial number. Figure 7 


.3 displays the data in graphic 


form. 
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MEAN CONTROL K'S ENTERED 



L = Low; H = High; N = Normal 





VERBEX 

MACINTOSH 




KEYBOARD 



Figure 7.2 



Mean Control K 7 s Entered vs. 
Env i r onmen t a 1 Cond i t ions 
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TABLE 7.4 



MEAN NUMBER OF COMMANDS ENTERED 
TRIAL NUMBERS 

Input Std Std Sta Sta 



Device 


1 rst 


Dev 


2nd 


Dev 


3rd 


Dev 


4 tn 


Dev 


Ver bex 


60.6 


21 . 1 


51 .5 


19.4 


60.1 


29.2 


55.3 


28.4 


Macintosh 


27.9 


9.2 


38.6 


15.4 


47.0 


7.6 


51 .3 


12.2 


Keyboard 


63.7 


23. 1 


76.2 


20.3 


73.2 


16.2 


83.2 


20.1 


Table 


7.5 summarizes 


the Mean Number of 


Control 


,_K ' s entered 


based on the Trie 


1 Number. 


Figure 7.4 displays 


the data in graphic 



form. 

TABLE 7.5 

MEAN NUMBER OF CONTROL K'S ENTERED 
TRIAL NUMBERS 

Input Std Std Std Std 



Device 


1 rst 


Dev 


2nd 


Dev 


3rd 


Dev 


4th 


Dev 


Verbex 


8.7 


oo 


6.8 


4.1 


3.6 


2.3 


4.5 


3.4 


Macintosh 


7.2 


3.7 


7.2 


3.6 


6.9 


3.7 


6.4 


3.4 


Keyboard 


8.1 


5.5 


9.0 


8.3 


6.1 


4.3 


5.2 


3.4 
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MEAN COMMANDS ENTERED 





VERBEX 

MACINTOSH 




KEYBOARD 



Figure 7.3 Mean Commands Entered vs. 
Trial Numbers 
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MEAN CONTROL K'S ENTERED 





VERBEX 

MACINTOSH 

KEYBOARD 



Figure 7.4 Mean Control K " s Entered vs. 
Trial Numbers 
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C. STATISTICAL METHODS 



Once the data front the experiment was compiled, two-way analysis 
of variance techniques helped evaluate the results. Two statistical 
analysis software proqrams resident in the IBM 3033 mainframe 
computer were used to facilitate the calculations. The first was 
MIKITAB, and the second was the Statistical Analysis System 
(SAS) . 



In testing analysis of variance the following two assumptions 
must be made: 1) the populations from which the samples came can 
be approximated clcsely with normal distributions, and 2) they 
all have the same variance. [Ref. 2C] For each case, the hypothesis 
v/as tested at a confidence level of 95 percent. Additionally, for 
each environmental condition (i.e., ncise/light level), a ratio of 
the number of errors that can be expected per the number of valid 
commands entered was calculated. This v/as computed for each of 
the three input devices. Finally, an analysis using SAS was 
conducted as a backup for some of the data that v/as analyzed 
using the MIMTAB program. [Ref. 21] 



D. ANALYSIS 

1. Introouction 

As with the statistical analysis, it is desirable to use 
as many data samples as possible. Since it v/as difficult to 
commit a large number of subjects to the experinent, each of the 
24 subjects participated in four separate sessions, thereby 
totaling 96 samples (VERBEX 3000 and Macintosh). Again, the 
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keyboard data v/as that which w as compiled from the VJr ight/Manscn 
exper iment. 

It was now important to ensure that the scenarios were 
varied enough to be an "independent session" for the subjects, so 
that no unfair advantage would occur during a trial run. The 
two-way analysis of variance v/as obtained between the number of 
commands entered and the different scenarios used. Also, a two-way 
analysis was conducted between the number of Control_K's entered 
and the different scenarios used. The result v/as the same for 
each, which indicated that neither one exhibited any significant 
difference. It is therefore reasonable to assume that the experiment 
consisted of 96 individual sessions. The Wr ight/Manson experiment 
consisted of 13 keyboard operators v/ho also played the identical 
set of 4 different scenarios (totalling 52) . The combined total 
is then 148 sessions (samples) for this experiment. 

2 . Best Environment for Each Input Method 

It v/as expected that each input device v/ould do better 
under a specific combination of environmental conditions. For two 
of three devices the results also showed that the combination 
that provided the greatest number of valid commands to be entered, 
also provided the fev/est number of errors (i.e., Control_K ' s ) . 
This is a desired situation and is reasonably intuitive. What 
follows is a brief description of the best environment found for 
each particular input method. See Figure 7.5 for a listing of the 
ratios (errors per valid commands) for each device. 
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VERBEX 



MACINTOSH 



KEYBOARD 



LOW/ 

NORMAL 



LOW/ 

LOW 



HIGH/ 

NORMAL 



HIGH/ 

LOW 




Figure 7.5 Ratio of Expected Errors Per Valid 
Command Under the Environmental 
Conditions Specified 
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For the VERBEX 3000 system the greatest number of valic 
commands were entered under a low noise, low light condition. 
Concurrently, the fewest errors were entered under these exact 
conditions. Based on the F ratios calculates after performing a 
two-way analysis of variance (at the S5 percent level), there v/as 
NO significant difference between the number of commands entered 
and the environmental conditions. The same result was realized 
(NO significant difference) between the number of Control_K's 
(errors inputed) and the environmental conditions. That is, 
although there is a "lowest ratio" for the VERBEX 3000 system, 
there is NO significant difference between the number of commands 
entered or errors entered, and the environmental conditions under 
which it was used. As expected, the lowest ratio (errors per 
valid command entries) came under the same lev/ noise, lo v; light 
condition. (See Appendix E for actual Statistical Data.) 

For the keyboard system, the highest number of valid 
commands were entered under tv/o conditions: 1) lew noise and 
normal lighting, and 2) low noise and lew lighting. Here again, 
the fewest number of errors were made under low noise and normal 
lighting conditions. Like the VERBEX 3000, the F ratio calculated 
from the two-way analysis of variance revealed NO significant 
difference between the number of commands entered (and the number 
of Control_K's entered) and the environmental condition. Never- 
theless, the lov/est ratio occurred under a low noise, normal 
lighting condition. 
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For the Macintosh* the highest number of valid commands 
were entered under high noise, low lighting conditions. The 
fewest number of errors were entered under low noise, lev/ light 
conditions. As with the previous input devices. NO significant 
difference was revealed between the number of commands entered 
(or the number of errors entered) and the environmental condition 
used. Here, the lowest ratio occurred while operating under low 
noise and low lighting conditions. 

3 . Best Inout Method for Each Environment 

This question was answered using a two-way analysis of 
variance. Each environmental condition (i.e.» noise/light) should 
reveal a desired input device which yields the largest number of 
commands, and also the fewest number of errors. As in the previous 
question, the ratio of errors to valid commands also revealed 
some practical insight into which device may be preferable when 
using a particular noise/light condition. The two hypotheses 
stated that the number of commands entered (and the number of 
Control_K's committed) would be equal for all three input devices. 

Conclusive results (based on the F ratio at 95 percent) 
show there IS a significant difference between the number of 
commands entered and the input devices. In all four noise/light 
conditions the keyboard operators entered more commands than all 
the other subjects. This was not the case, however, when a measure- 
ment of the number of Control_K's committed was taken. Here, the 
two-way analysis of variance revealed no significant difference 
among any noise/light condition and the input devices. Even on 
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examining the raw data it was shov/n that NO clear advantage 
existed for any particular device. The calculated ratios (see 
Figure 7.5) displayed the following advantages under each environ- 
mental condition: 

BEST INPUT METHOD 



1. Low Noise / Normal Light KEYBOARD 

2. Low Noise / Low Light VERBEX 

3. High Noise / Normal Light KEYBOARD 

4. High Noise / Low Light VERBEX 



4 . Best Overall Input Method 

This question was answered using a cne-way analysis of 
variance. The F ratio was again calculated at a 95 percent confidence 
interval, and the hypotheses were the same as the ones described 
in the previous question. 

The results revealed a significant difference between the 
mean number of commands entered and the input devices, thereby 
rejecting the null hypothesis. The raw data showed that the 
keyboard operators entered more commands than the operators of 
the other devices. The VERBEX 3000 players were only slightly 
Icv/er, while the Macintosh players entered an average of 43 
percent fewer commands per session. 

As far as the number of errors committed was concerned, 
the one-way analysis of variance showed NO significant difference; 
thereby accepting the null hypothesis that the number of Control_K's 
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entered by all three input cevices w as equal. The rav/ data showed 
only a difference of 2 or 3 errors amongst input devices per each 
session. The VEBBEX 3000 had the fewest (an average of 5.0), next 
was the Macintosh (with 6.9), and finally the keyboard (with 8.1). 

5 . Possible Learning Curves 

It was interesting to note that NO significant learning 
curve took place between the subjects when considering either 
the number of commands entered, or the number cf Control_K's 
committed. It is v/orth mentioning, however, that the Macintosh 
subjects increased the number of commands entered for each succeeding 
trial (session) . This can be seen graphically depicted in Figure 
7.3. The reverse w as not the same for the number of Control_K's 
that the subjects entered, and again, neither w as it significant 
(this can be seen graphically in Figure 7.4). 

E. SUMMARY 

To summarize, there were few significant results. Even with 
the addea benefit of looking at the rav/ cata, not all the questions 
were answered unequivocally. None of the three input devices 
could be labeled as a clear-cut winner over any of the other 
devices. Each of the devices performed best given a specific set 
of environmental conditions. Oddly enough, the V HR BEX 3000 and 
Macintosh both performed best in the Low Kcise/Lcw Light conditions, 
and the keyboard performed the best in the High Noise/Kormal 
Light conditions. Without regard to the Noise/Light conditions, 
the VERBEX 3000 system had the lov/est ratio (i.e., error to cormana) , 
with the keyboard a close second (.102 to .112 respectively). The 
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Macintosh entereo fewer conxands than the others, ano was ranked 
second v/hen measuring the nur.bers of errors comir.itted. Additionally, 
no significant learning took place during the four separate 
sessions that each individual participated in during the e:cper irrent . 
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VIII. 



CONCLUSIONS AND RECOMMENDATIONS 



A. INTRODUCTION 

The objective of this thesis v; as to evaluate Different input 
interfaces to the KWISS wargaming scenarios resident on the VAX 
11/780 computer. The two hypotheses were that the mean number of 
commands entered and the mean number of errors committed (using 
any of the three tested input devices) would be equivalent. While 
the objectives of the thesis were successfully established on the 
completion of the experiment, no single input device can be 
designated as the "better" of the three. 

B. CONCLUSIONS 

Before the experiment, the authors of this thesis anticipated 
that the voice system would outperform the other devices. By the 
term "outperform", it is meant that the VERBEX 3000 subjects 
would enter significantly more commands and commit fewer overall 
errors than the other devices. It was expected that the Macintosh 
v/ould outperform the keyboard simply by virtue of its current 
technology. In general, the analysis of the data did not confirm 
these expected outcomes. 

Many reasons can be given to explain these unexpected results. 
As far as the keyboard v/as concerned, most of the subjects wasteo 
little time in moving their keyboards as close as possible to 
their data input (CRT) screen in the low light condition. Using 
the video screens inherent glcw helped considerably in locating 
the correct keys; especially for the non-typists. Also, a post 
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experiment survey revealed that the keyboard operators were 
experienced typists by and large. Ev this it is meant that at 
worst cost of the subjects v/ere able to use both hands while only 
occasionally looking at the keys. Ev implication, military officer 
postgraduate students are collectively more likely to have had 
typing experience (i.e., used typing skills during their lengthy 
education/training) than the general population. Their average 
lifetime exposure to keyboard technology is high, and again this 
particular group was above the norm. [Ref . 2] On the other 
hand, none of the subjects had prior experience v/ith either the 
continuous voice system, and only a few subjects had prior experience 
v/ith the Macintosh microcomputer system. This brings into focus 
the implications of the necessary training involved for each 
device. The VERBEX 3000 subjects trained v/ith the voice system an 
average of six hours only one week before the experiment. Considering 
that it takes years of frequent typing to become proficient, the 
VERBEX 3000 subjects did v/ell to keep up with the keyboard operators. 
The Macintosh subjects practiced for about 35 minutes each, 
concentrating totally on developing the motor skills necessary to 
maneuver the arrow using the mouse peripheral. In retrospect, it 
is felt that they would have performed considerably better if 
they had been as proficient (through additional training) in the 
use of the mouse as the keyboard subjects v/ere in using their 
acquired typing skills. The raw data from the Macintosh players 
showed a slight increase in the mean number of commands entered 
for each individuals' succeeding trial. Although it was not a 
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significant increase, the same statement could not be mace for 
the other devices. 

When looking at the ratios (see Figure 7.5), each cevice 
performed well under certain noise/light conditions. The VERB EX 
3C00 and the Macintosh had the lowest error to command ratio 
under the low noise/lcw light conditions. Not surprisingly, the 
keyboard operators had the lowest ratio under the high noise/normal 
light conditions. For the keyboard and VERBEX operators, each hac 
the lowest error to command ratio in two of the four environmental 
conditions. Looking specifically at the "light conditions" (without 
regard to noise) , the VERBEX 3000 system had a lower ratio than 
the other devices. Similarly, the keyboard users attained their 
lov/est ratio under "normal light conditions". Considering the 
inexperience of the VERBEX 3000 operators, they did extremely 
well in the most likely high stress, wartime environment (high 
noise, lov; light). The error to command ratio w as .015 lower than 
the second best, keyboard. 

Although we can logically conclude that the keyboard die 
perform better than the other devices, it would be illogical to 
conclude that the other input devices do not warrant further 
research. Just looking specifically at noise conaitions (without 
regard to the lights) the VERBEX 3000 system had a lower ratio 
than the other devices. Considering the inexperience of the 
VERBEX 3000 operators, their ratio was only .004 higher than the 
keyboard subjects in the most likely high stress wartime environment 
(that of high noise and lov/ light conditions) . 
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Another area analyzed was the presence of any learning trends 
the subjects may have received from previous sessions. Hone of 
the subjects performed significantly better over the four trials. 
While this was not necessarily an encouraging trend, this result 
supports the independence of each session, thereby making the 
sample size of the experiment itself more valid. 

One final comment should be made about the likelihood that 
inexperienced operators will often be at the controls of the 
input media in an academic or training environment. For example, 
a senior officer (i.e.» 0-5 or above) attending the Naval War 
College will play similar war games. It is highly unlikely that 
an individual will have had prior experience using speech or 
"menu-oriented" systems for input communication. It car. therefore 
be concluded that with a minimum of introduction to these devices, 
almost equal performance can be attained to that cf the experienced 
typist if the syntax required of the game commands is also well 
learned by the novice user. 

C. RECOMMENDATIONS 

It is the opinion of the authors that further study is justified 
concerning the potential of the three input devices. Eefore 
another experiment is performed, however, it is recommenced that 
sufficient training of the subjects be conducted. It was felt 
that the Macintosh subjects would have performed significantly 
better with more extensive training and dedicated practice sessions. 
This would have enabled them to become more familiar with the 
coordination required to effectively maneuver the cursor with the 
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mouse. The problem with this proposal is the time constraint 
imposed on both the planning and execution of the experiment. 
Optimally, all subjects should be equally skilled in the use of 
the particular input oevice. 

Additionally , the software interface program for the Macintosh 
should be modified to facilitate a quick, efficient and accurate 
mode of input for the user. This modification would require a 
more natural arrangement of commands to allow an easier interaction 
with the sequence of these commands required by the syntax design 
of the software. 

Acquisition costs for these devices and any study of cost/benefit 
relationships would provide additional fertile ground for future 
research. Current downward trends in hardware costs, and a general 
lowering of costs for all technologies which are driven by micro- 
computer advances in the marketplace, cause a "concept of costs" 
for these systems to be well worth consideration. 

NWISS should be used as a medium for any further experimentation. 
The necessary peripherals are all established, and it provides an 
accurate simulation of a real-time environment. Varying environ- 
mental conditions are easily simulated in the C3 laboratory, and 
the accessibility limitation makes it an ideal place to conduct 
further experiments. 

In conclusion, future experiments should include the previously 
suggested modification to the Macintosh software and the most 
advanced "touch screen" and voice technology available. It is 



felt that these three devices have the greatest potential to meet 
the requirements for an efficient input medium. 



D. SUMMARY 

Han's zeal to accomplish the ideal "man-machine" interaction 
has allowed significant progress to occur regarding input devices. 
The keyboard was first, and has been dependable but restrictive; 
sometimes to the point of being inconvenient. Cuick to approach 
popularity were several "non-traditicnal" methods; one being the 
voice system. The discrete voice system is unnatural and hard to 
use for most users, while the continuous system is easier to use 
and more convenient for the jobs requiring user input and manual 
labor simultaneously. Two other methods are the "pull-down" menu 
anc the "touch screen" technologies. Both have been successful 
even in their early stages of development. 

Man is quickly approaching the doorstep of Fifth Generation 
computer technology. Therein lie "intelligent systems" which will 
give increasing flexibility to input devices of the future. 
Unfortunately, there can be no "perfect" universal input method 
due primarily to the varied requirements and restrictions innate 
to each individual job. Nevertheless, it follows that continued 
research should be aimed at producing a body of knowledge which 
would allow us to pick the best method of computer interaction in 
a specific environment. Although it is recognized that computer 
speeds today are restricted from full use because of the slowness 
of interface hardware, the rate of progress in developing high 
speed CPUs is compounding the problem. Therein lies the impetus 
to continue toward the development of better input media. 
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APPENDIX A 



SCENARIO BRIEFINGS 



SCENARIO A 



SITUATION: Hostilities have already been declared, and Orange 
forces are providing escort for a large convoy of cargo ships 
carrying nuclear/chemical weapons. 

FRIENDLY FORCES: You have available one Aircraft Carrier, 2 Knox 
class destroyers, a Los Angeles class submarine (positioneo north 
of the convoy) and 2 flights of 4 P3C aircraft. 

ENEMY FORCES: Orange forces have 8 ships escorting 10 merchant 
ships in the convoy. No air assets are presently available, or 
expected. 

MISSION: Destroy all Orange combatants without inflicting damage 
to any cargo vessels (due to the nature of their cargo) . 



MOE: POSITIVE 

— number of weapons fired during the session 

— number of combatants identified, hit and damagec/sunk 

NEGATIVE 

— number of merchant ships hit/sunk 

— number of Elue Aircraft lost 

— number of Elue vessels hit/sunk 
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SCENARIO B 



SITUATION: Appr oxiirately 20 Orange merchant vessels are anchored 
in port waiting to unload various cargo. Hostilities are ir.m.inent, 

FRIENDLY FORCES: Blue forces include 2 P3C aircraft anc. two 

groups of 5 A6E's. An aircraft carrier is available to supply 
further air assets if deemed necessary. 

ENEMY FORCES: Location and makeup of Orange forces is undetermined 

due to poor weather/intelligence . 

MISSION: Destroy as many merchant ships as possible, and attack 
combatants only if fired upon. 



MOB: POSITIVE 



number 


of 


weapons fired 








number 


of 


merchant ships icentif 


iec , 


hit 


and damaged 


number 


of 


combatants identified, 


hit 


end 


damaged 



NEGATIVE 



— number 



— number 



of Blue forces damaged/lost 
of merchant ships remaining 
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SCENARIO C 



SITUATION: There are confirmed reports that large numbers of 
Orange Combatant units have left port, are running Kf'CON silent 
and are unlocatable at this tine. There are f r iendly/neutral 
forces in the neighboring area and tension is rising. No hostilities 
have been declared as yet. 

FRIENDLY FORCES: 1 P3C aircraft and 4 F14A aircraft are available. 
Other assets are in the area, but are not uncer your direct 
control. Any merchant ships encountered are not to be consicered 
hostile . 

MISSION: Search for possible contacts in the area from 35N to 
42N, and from 174E to 180E (or 164E to 170E depending or. the 
oceanic displacement). Locate, identify and designate as many 
vessels as possible as either enemy, friendly or neutral. Center 
the plot upon the P3C at the start of the session, and weapons 
will remain tight unless fired upon. 



HOE: POSITIVE 



— number of Orange forces properly identified 

— number of Elue forces properly identified 

— number of Neutral forces properly identified 

NEGATIVE 

— number of forces improperly designated 

— number of forces remaining not found/properly designated 
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SCENARIO D 



SITUATION: An Orancje SAG is 100 NH directly north of your pos] 
and you are tasked to intercept anc visually identify it. Hostilj 
have not been ceclared as yet, but are expected to break out s 

FRIENDLY FORCES: One Aircraft Carrier (with its associateo airerc 
3 escort ships ana a submarine. 

ENEMY FORCES: The strength of the Orange forces is unknown, 
they are not expected to have any aircraft launched at this t 

MISSION: Launch carrier based assets to search/identify 
suspected Orange SAG. All v/eapons will remain tight unless j 
upon. 



MOE: POSITIVE 



— number 

— number 

— number 



Of 


aircraft successfully 


launched 


of 


enemy 


assets 


properly 


iaent if ied 


of 


enemy 


assets 


damaged 


(only if fired upon) 



NEGATIVE 



— number 

— number 

— number 



of Orange forces not designated as enemy 
of Elue ships hit/lcst 
of Elue aircraft lost 
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APPENDIX B 



EXPERIMENT TEAM MEMBERSHIP/SCHEDULE 
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APPENDIX C 



SUBJECT ADMINISTRATIVE BRIEFING 



NOTE: The follov/ing administrative briefing was initially given 
to each subject prior tc the beginning of the experiment. Key 
points were re-emphasized at each session as applicable. 



1. The sessions would be approximately 50 minutes ir. length, eacn 
subject would be scheduled for four individual sessions and a 
master schedule of all sessions was posted on CDR. Stewart's 
office door for reference, and to facilitate changes throughout 
the experiment. 

2. Subjects were asked to wait outside the laboratory until told 
to enter. This precaution was taken in order to preclude an 
inadvertent disturbance in the environment if a previous group 
session was still in progress. 

3. The voice patterns for the scheduled VERBEX 3000 subject were 
already loaded to avoid any unnecessary delay. The user simply 
had to set the gain in order to be ready to start the session. 

4. The noise/light conditions for the particular session were 
explained and set. 

5. The subject was provided with the scenario for the session 
and all key points were verbally emphasized. 

6. All subjects were advised to ask for help during the session 
if any problems were encountered. 

7. The subjects were asked not to discuss any of the scenarios 
following their sessions until the experiment was completed. 

8. Each group was reminded of their next scheduled laboratory 
period following the completion of a session. 
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APPENDIX D 



EXAMPLES OF COMMAND SEQUENCES 



- "Display ... Radius 2 5 0" 

- "Execute BAIRASW" 

- "Execute F14STRCAP" 

- "For Kitty ... Launch ... 2 A7E A6E ...(course) 2 3 5... (speed) 

4 3 0 ... (altitude) 1500" 

Fit. Plan: "Mission Strike" 

Fit. Plan: "Proceed position ... 4 5 N ... 1 1 5 - 2 0 E" 
Fit. Plan: "Load ... (equipment) 3 Shrik ... 6 MK82" 

Fit. Plan: "Stop" 

- "Designate Enemy ... (track) E E 0 0 5" 

- "Display ... Place Circle Force ... M P 6 1 5 (radius) 50" 

- "For V A 0 1 0 ...Weapons Free Enemy All" 

- "For S H 2 0 0 ... Earrier ...(position) 1 5-20 N ... 125 
- 1 5 E ... (bearing) 1 3 5 ... (distance) 4 5 ... (using) 

5 S Q 6 2 ... (spacing) 4" 



NOTE: Words in (parenthesis) are prompts that appear on the 
speech unit. 
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APPENDIX E 



INDIVIDUAL DATA 
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APPENDIX E (cont.) 
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APPENDIX E (cont.) 
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APPENDIX F 



NWISS/VERBEX 3000 USERS GUIDE 



INTRODUCTION: 



This users guide was prepared to facilitate cognizant individuals 
in training their individual speech patterns to the Verbex 30 OC 
voice terminal. The unit has a maximum vocabulary capacity for 
360 words/utterances, 150 of which will be trained in order to 
interface v/ar gaming scenarios resident on the VAX 11/7 80 computer. 

The guide is divided into the two (2) separate sections that 
are representative of each actual phase of training. The first 
phase is ENROLLMENT, which allows each word/utterance to be 
individually and repeatedly spoken. The second phase is TRAINING, 
which incorporates the repeated use of the phraseology necessary 
for wargame scenario interaction. 

What follows is a list of specific procedures used to enable 
the sytem for both the ENROLLMENT and TRAINING phases of the 
voice experiment: 



1. The machines (Verbex, ADM 31, VT-102, and the speech unit) are 
connected using the proper adapter as shov/n in Figure tfl. (Figure 
#2 shows the configuration to run NWISS on the VERBEX 3000) . 
The small machine to the right of the VT-102 has "VERBEX 
3000" printed on the top, and will hereafter be referrec to 
as the "Speech Unit", so as to not be confused with the large 
VERBEX 3000 unit located just to the left of the table. The 
following specific procedures should be utilized to effect 
proper set-up: 



A. Turn the ADM 31, VERBEX, and VT-102 to the power-on position 

(see Figure #1) . NOTE : Turning the ADM 31 on is optional 
for both phases. 

B. The VT-102 and the speech unit will automatically run 

through a self-test (using the alphabet) . Upon completion 
of this procedure, the speech unit will read: " Booting 

VERBEX 3000 ". NOTE: The VT-102 screen will be a scrambled/ 
garbled alphabet on occasion; this is still considered to 
be normal operation and will not effect the follcw-on 
VERBEX enrollment/training phases. 
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C. Now complete the following steps to enable the system: 

1. Press the SET-UP key on the VT-102. 

2. Press the Set Up h / B (#5) key to set-up "B". 

3. Press the Transmit Speed (#7) key press slowly a no. 

repeatedly to scroll to 4800. 

4. Press the P.eceive Speed (#8) key press slowly and 

repeatedly to scroll to 4800. 

5. Press the SET-UP key to exit. 

D. Hit the RETURN KEY (of VT-102) this will move the 

cursor to the left side of the screen. 

E. With the cursor on the left side of the (VT-102) screen enter 

the following commands sequentially, and wait for the 
computer to respond (as shown in the output column) 
between entries: 



ENTER 


OUTPUT 


dir <return> 


directory listing 


ed .asd 


to use the edit . 


n 


to number lines 


1,$P 


to print lines 



F. If KWISS :w:/NWISS is on line 1: Enter " x ", and continue 

with step letter I. 

G. If NWISS :w:/NWISS is not on line 1, enter: 

1. 2m0 (assuming it is on line two) 

2. l,$p 

3. x ( this saves the new order ) 

H. The prompt on the VT-102 should be " - " (a dash) . 
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I. With a " - " as a prompt, enter: nwiss/nwiss 

J. The screen shoulc read: 

Monitor: Version 2.0 
Copyright (c) 1983, Exxon Corp. 

Galcon 3.3 - Grasp 3.1 

and the speech unit will read: " Enter user ID = ". 



2. From the speech unit enter " 00 ", then press the enter key. 
This will open the supervisor mode. ( NOTE : the Enter Key must be 
pressed after each entry from the key pad on the speech unit) . If 
you realize you have selected an incorrect number for input prior 
to pressing " ENTER ", you may press " CLR " (the bottom right 
key on the VERBEX keyboard) and then press the correct number to 
be reentered. The " Please Wait " light will illuminate, and 
after a minute or two the speech unit will read: "Add New User?". 

3. You will now begin to scroll through Menu #1 (see Menu Listings) . 
Press the NO button to scroll through the Menu for familiarization 
purposes . 

4. To begin the ENROLLMENT phase (approx. 20 min.) Press YES 
when the menu reads "Add New User?". 

5. The speech unit will now read "Enter New User ID =". Enter 
your user number and press ENTER. NOTE: Be sure not to enter a 
number already on the enclosed list of used numbers, (see MENU 
GUIDE) . 

6. Put the Headset on v/ith the Mike on the left side (see Figure 
# 3). MIKE POSITION IS CRITICAL !! Position the Mike only a 
little to the left of center, and slightly lower than the center 
of the mouth. It should be no more than 1/2 inch (i.e. about one 
thumbswidth) from the lips without touching them. Periodically 
check the microphone for proper placement in order to insure 
consistency. 

7. When the microphone is in place and you are ready, start the 
amplifier and the tape deck in Bay #2 (in order to interject 
crowd noise at 65 db into the training), using instructions 
attached to the tape deck itself. 

8. The speech unit will prompt you to "SET the GAIN". 

It will read "Gain Now at 1, Yes to Set Gain". 

(NOTE: A gain of 1-3 is OK — under normal voice levels, a 
gain of 2 - 3 will be maintained ) . 
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9. When reacy to proceed, press the YES button and perform the 
following steps: 



A. Follow prompt (ie. say "One, two, three, four, 
testing" ) . 

E. The speech unit will now reao: "Gain is set to (1-4), 

YES to set gain". 

C. Press the YES button. 

D. Follow the prompts as before. 

E. The speech unit will now read: "Gain is set to 3, YES to 
set gain". 

F. Press the YES button. 

G. Eventually the speech unit will read: "YES to sample 

backrouncl noise". 

H. Press the YES button. 



I. The speech unit will now read: "Remain Silent". At this 

point do not make any additional noise other than normal 
breathing (lasts about 2-3 seconds) . 

J. The speech unit will read: "Yes to sample silence". 

K. Press the YES button. 



L. 



The speech unit will now read: 
do not make any additional noise 
silence of the environment. 



"Remain Silent". Again, 
while it is sampling the 



M. The speech unit will now read: "Gain = (1-4). YES to 

proceed" . 



10. NOTE: 

A. If the gain is 1-3, do not press YES until you are ready 
to proceed with the exercise. Be aware that the STOF button 
can be used at anytime throughout the training cycle for 
one of three (3) possible reasons: 

1. If you would just like to take a break for a fev; minutes, 
press the STOP button and you will be prompted to 
"Continue Training?". Simply press YES when you are 
ready to continue. 
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2. If you have mace a mistake on an utterance, press the 
STOP button. Use the NO button to scroll through the 
menu (#2) until you read "Reject Last Utterance?". 
Pressing the YES button will accept that command and 
im.nec iatelv continue with the next word or phrase. 

3. If you feel what you have dene is ccr.pletely unsal- 
vageable to this point, press STOP. Scroll through 
the menu (#2) until you read "Logout?". Press YES, 
and if you are in the ENROLLMENT phase, continue with 
Step #2. If you are in the TRAINING phase, continue 
with Step # 13 a. 

N OT E.; 



Get comfortable before you start. Take breaks if you need them, 
and strive to be consistent with your voice modulation. See 
Enclosure # 1 for a complete list of the words you will see 
throughout the session. The words will be in order of appearance, 
and the "less than obvious" annunciations will be annotated for 
standardization purposes. 

If the gain =4, it is best to start again. One way to do this is 
to continue as normal until the actual cycle has started prompting 
you with the first word of the session. At this point press the 
STOP button and scroll through the menu until you read "Logout?". 
Press YES. The speech unit will read: "Exit (your ID £)", and 

will then read: "Logout supervisor", followed by: "Enter User 

ID". Continue from Step #2 using your same initial ID number when 
responding to the "Add New User?" prompt. 

11. Upon completion of the enrollment phase you v/ill see "Enrollment 
Done. YES to Save". Push the YES button and wait for the "Patterns 
Saved. YES to Train" prompt to appear. Nov; push the following 
buttons in the sequence below: 

A. Push the STOP button. 

B. "Resume Enrollment?"; press the NO button. 

C. "Reject Last Utterance?"; press the NO button. 

D. "Save Patterns?"; press the NO button. 

E. "Logout?"; press the YES button. 

12. This concludes the ENROLLMENT phase of the individual voice 
training cycle. 
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13. The next phase is TRAINING and it includes ten (10) different 
sections. You will be asked to train different combinations 
of the v/orcs you used in the ENROLLMENT phase. The names and 



approximate "time to 


complete" are 


listed 


below : 






A. 


NWISSGRAM1 


: 38 min. 


6. 


FIRE : 


17 


min . 


B. 


FORCE 


: 27 min 


7. 


SHIPORDER : 


24 


min . 


C. 


DISPLAY 


: 13 min 


8. 


AIRORDER : 


31 


min . 


D. 


POSITION 


: 6 min. 


9. 


DIGITS : 


S 


min • 


E. 


TRACK 


: 7 min. 


1C. 


LAUNCH : 


32 


min . 



NOTE: You may do these in any order. 



The following procedures should be adhered to for each section of 
the TRAINING phase: 



A. When the speech unit reads: "Enter user ID", you will 

enter: "00". This procedure logs on the "supervisor" . 

B. The speech unit will then read "Add New User?" (as part of 
menu #1) . 

C. Scroll through the menu with the NC button. Stop scrolling 

at: "Retrain Old User?". 

D. Press YES. You will now be prompted for your ID 
number. Enter the same number as before. The speech unit 
v/ ill now read: "Loading Patterns for user ID (your ID #)". 

E. After a minute or so, the speech unit will read: "All 

Grammars?". Always respond by pressing the NO button. 

NOTE: If you press YES, you v/ill be asking for a long exercise. 



F. The speech unit will read a menu (#4) of all ten (10) 
selections. Scroll through it with the NO button until 
you find a section you wish to train. 

G. When you find it, press the YES button. 



H. The speech unit will prompt you to Set the Gain. Do 
this exactly as before (Step # 8). Remember, be ready to 
start speaking immediately after pressing the YES button 
in response to the "YES to proceed" prompt. Examples of 
word combinations are listed in Enclosures (2) . 
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I. As in the ENROLLMENT phase, you v/ill be able to use 
the STOP button for the same reasons as stated in 

Step #10. 

J. At the completion of each of the ten (10) training sections, 

the speech unit will read: "YES to save". 

K. Press the YES button. 

L. The speech unit will now read: "Patterns Savec. YES 

to Test". 

K. Press the YES button. 



** NOTE ** 



Test Sessions: 



1. For each completed and saved training section (except 
"DISPLAY"), you will be prompted to say one word or 
phrase; usually it will be the word "Control_K". 
Respond using the same annunciation and voice modulation 
as you die during training. The speech unit will 
score your training ability on a scale from 0 to 100. 
Acceptable scores range from 80 to 100. Lower scores 
are an indication that possible retraining is warrantee. 

2. After the DISPLAY section is completed and saved, a 
similar but longer test v/ill be given. Each response 
v/ill be scored, as well as an average score given for 
that particular section. The test will take about 1 
minute . 



14. At the completion of each test session, the speech unit v/ill 
read: "Score = . YES for Menu". 

15. Press the YES button. The speech unit v/ill now reac: "Exits 
", then: "Add New User?" (Menu #1). 

16. At this point if you would like to continue with another 
section of the TRAINING Phase, simply scroll through the Menu 
(using the NO button) until the speech unit reads: "Retrain Old 
User?". Nov/ continue with step 13d. 

17. If instead you would like to log-out at this point, scroll 
through the Menu until the speech unit reads: "Logout?". Fress 
the YES button. The speech unit v/ill now read (in respective 
order): "Exit ID (your ID #) ; "Logout supervisor" ("00"); and 
"Enter User ID". 

18. If you decide to continue and interface with the NWISS 
wargarr.ing scenarios, the system set-up will have to be modified 
in accordance v/ith Figure #2, and then you can continue with step 
# 20 . 
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19. To shut the system dov;n, simply place ail or the individual 
power switches (On/Off switches) to the OFF position. 

20. With the VT-10 2 and the ADM 31 establish the initial set-up 
as per procedure 1 (A-J) . The speech unit should now read 
"Enter User ID:". Enter vcur ID number and cress ENTER. 

21. The speech unit will no w read "Loading Patterns for ID = 

22. You can set the gain (as per #8 - 1C ) either prior to or 
after the KIvISS is brought online. 

23. Upon completion of the came, press the STOP button. Scroll 
through the Menu #3, and follow steps 17 and 19. 
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HELPFUL HINTS 



1. Use the enclosed VJcrclist as a guide while going through the 
ENROLLMENT Phase. It would also be very beneficial to have studies 
it prior to attempting the phase, as the words are presented too 
fast to think about pronunciation prior to speaking ther. 

2. Sit comfortably; preferably directly in front of the speech unit. 

3. Be ready to push the STOP button if you make a mistake. 

4. Study the word list (Enclosure 1) prior to enrolling. Especially 
consider the v/ords in the "pronounce" column. Say then naturally. 
DO MOT pause for any word in the ENROLLMENT phase. The spaces in 
that column are to help determine the proper annunciation of the 
word. It will be necessary to pause in the TRAINING phase, but 
only long enough to distinguish between v/ords. 

For example , ("FOR KITTY"). 

5. In general, use the most natural pronunciation that comes to 
mind. 

6. Pronounce numbers which appear as part of fixed identifiers, 
in a natural manner (i.e. F14A as F Fourteen A; MK48 as Mark 
Forty-Eight, etc.). 

7. Fully pronounce digits which appear as part of variable strings 
(i.e. Altitude 2500 as Altitude Two Five Zero Zero, etc.). 

8. Be consistent with your pronunciations. 

9. The Select YES/NO light: 

A. This light is on the console of the speech unit. When it 
is illuminated it means press the YES or NO button as 
applicable. 

B. NOTE: This procedure is only true if the speech unit 
reads a line of a " Menu ". At all other times, only the 
YES button will be accepted. 
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MENU GUIDE 



NOTE; 



Do net use the following list of ID nuirbers: 
10 (inclusive) ,47 ,69 70 or 74 



00 th r ouch 



MENU 11 



MEMLJL2 



This menu is invoked by entering This menu is invoked by 

the supervisor mode (i.e. 00 into by use of the STOP 

the speech unit) button v/ithin ENROLLMENT 



Phase . 



Add New User? 

Test Old User? 
Re-Enroll Ole User? 
Re-Train Old User? 
List User? 

Delete Ole User? 



Resume Enrollment? 
Reject Last Utterance? 

Save Patterns? 
Logout? 



Logout? 
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MENU *3 



MENU #4 



Invoked by completing the test Invoked by commencement 

session in the TRAINING phase of the training 



Continue? 

Calibrate? 

Restart? 

Logout? 



All Grammars? 
NWISS1? 

Force? 

Display? 

Position? 

Track? 

Fire? 

Shiporder ? 
Airorder ? 
Digits? 
Launch? 
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WORDS TO BE TRAINED ON THE VERBEX 



1 . 


WORD 

Execute 


PRONOUNCE 


WORD 

25. W i 1 s o 


PRONOUNCE 

WILSON 


2. 


BAIRASW 


B AIR AS W 


26. Knox 


NGX 


3. 


BCAP2 


BEECAP2 


27. McCor 


McCCRMICK 


4. 


ECAP1 


BEECAP1 


28. Kitty 


KITTY 


5. 


F14STRCAP 


F FOURTEEN 
STRIKECAP 


29. VTO 

30. 0 


VEE TEE ZERi 
ZERO 


6. 


A6 STRIKE 


. A SIXSTRIKE 


31. 1 




7. 


Control_k 




32. 2 




8. 


Force 




33. 3 




9. 


Track 




34. 4 




10. 


Position 




35. 5 




11. 


Des ignate 




36. 6 




12. 


Neutral 




37. 7 




13. 


Fr iendly 




38. 8 




14. 


Enemy 




39. 9 


NINE (not 


15. 


Display 




40. VS0 


VEE S ZERO 


16. 


Bearing 




41. VH0 


VEE H ZERO 


17. 


For 




42. VE1 


VEE E 1 


18. 


IPO INTI 


1.1 


43. VE0 


VEE E ZERO 


19. 


Sprua 


SPRUANCE 


44. W0 


VEE W ZERO 


20. 


Misawa 


MISAWA 


45. VK0 


VEE K ZERO 


21. 


Losan 


LOS ANGELES 


46. VF0 


VEE F ZERO 


22. 


Rathb 


RATH BURN 


47. VA0 


VEE A ZERO 


23. 


Omaha 


OMAHA 


48. SHI 


SHI 


24. 


Wichi 


WICHITA 


49. MP6 


M PEE SIX 



niner ) 
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50. Vlad 




77. 


BA0 


BEE A ZERO 


51. Petro 


PETRO 


78. 


Tor.ihk 


TOhOHAWE 


52. Alexs 




79. 


Hrpon 


HARPOON 


53. Wonsa 


WARDS A 


80. 


Cruise 




54. Erase 




81. 


ASROC 


AS ROCK 


55. Plot 




82. 


MK46A 


MARK FORTYSIX A 


56. LOB 


LOB 


83. 


MK46 


MARK FORTYSIX 


57. ESM 


E S M 


84. 


MK4 8 


MARK FORTY EIGHT 


58. Sonar 




85. 


Torpedo 




86. Drop 




86. 


At 


AT (cne word) 


60. Cancel 




87. 


Range 




61. Grid 




88. 


Orders 




62. Circle 




89. 


Take 




63. XMAP.K 


X mark 


90. 


DECK 


D E C M 


64. All 




91. 


Off 




65. Place 




92. 


On 




66. Center 




93. 


RBOCK 


ARBOCE 


67. Radius 




94. 


Blip 




68. - 


TACK 


95. 


Launch 




69. N 


NORTH 


96. 


Weapons 




70. E 


EAST 


97. 


Free 




71. W 


WEST 


98. 


Submarine 




72. S 


SOUTH 


99. 


Surface 




73. BUO 


BEE YOU ZERO 


100. 


Air 




74. BSO 


BEE S ZERO 


101. 


Tight 




75. BPO 


BEE P ZERO 


10 2. 


EMCON 


MCOK 


76. BEO 


BEE E ZERO 


103. 


Surf 


SURF 



i m 



104. AIRS 


AIRS 


130. SSG62 


c 


S 0 S IXTYTNG 


105. AEW 


A E W 


131. SSG53 


S 


S G FIFTY THREE 


106. Silen 


SILENT 


132. SSQ47 


S 


S Q FORTY SEVEN 


107. F.adia 


RADIATE 


133. A6E 


A 


SIX E 


108. Fire 




134. A7E 


A 


SEVEN E 


109. Station 




135. E2C 


E 


TNG C 


110. Periscope 


136. EA3B 


E 


A THREE B 


111. Depth 




137. EA6B 


E 


A SIX B 


112. Course 




138. FI 4 A 


F 


FOURTEEN A 


113. Speed 




139. F14T 


F 


FOURTEEN T 


114. Proceed 




140. KA6D 


K 


A SIX D 


115. Barrier 




141. P3C 


PEE THREE CEE 


116. Mission 




142. S3A 


S 


THREE A 


117. ASW 


ASW 


143. SH3H 


S 


B THREE H 


118. SURCAP 


SIRCAP 


144. SH2F 


s 


B TWO F 


119. Search 




145. HARP 


HARP 


120. CAP 




146. MK82 


MARK EIGHTYTWG 


121. Airtanker 


147. MK83 


MARK EIGHTYTRKEE 


122. STRCAF 


STRIKECAP 


148. MK84 


MARK EIGHTYFOUR 


123. Strike 




149. Phenx 


PHEENIX 


124. Cover 




150. Shrik 


SHRIKE 


125. Altitude 




151. Spar 


SPARROW 


126. Bingo 




152. Swdr 


SIDEWINDER 


127. Refuel 




153. Wall i 


WALLEYE 


128. VK00 


VEE K ZEROZERO 


154. Load 






129. SQ538 


S Q FIVE 
THIRTY El GET 


155. Stop 







ENCLOSURE (1) 
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TRAINING PHASE: EXAMPLES OF WORD COMBINATIONS 



MWISSG RM I 

Bearing Track 
ES-COO 
VT-C01 
BAIRASW 

Designate Neutral 
Display 

Execute F14STRCAP 

VF-068 

Position 

SH-105 

For lFointl 

For Knox 

P QS I T IQN 

_ ~> C 

5- 5 
2 8 VI 
1 8 E 
13 4 
4 0 S 
7 9- 

6- 2 
- 35 
9 7- 



FQEQE 

VE-100 

VF-101 

VK-003 

VF-C25 

VH-C04 

Spruance 

A 1 e k s 

Vlad 

Misawa 

Wilson 

MP-600 

Ratbbu rn 

TRACK 

BU-059 

BU-009 

BE-026 

BA-034 

BP-047 

BS-012 

BE-039 

BA-057 

BP-065 

BS-017 

ENCLOSURE (2) 



DISPLAY 

Radius 3 4 
19 5 

Erase LOB ESN 
Plot LOB Sonar 
Cancel Circle All 
Place Grid Track 
Center Force 
Cancel Circle Posit 
Place Grio Force 
Place XKark Force 
Place XMark Posit 
Center Position 

FI£E 

8 4 9 

2 Tomhk Cruise 
Bearing 5 7 

3 ASROC Torpecio 
At Aleks 

At Wonsa 
15 0 

2 Ton;hk Cruise 

4 Rrpn Cruise 

5 HK46A Torpedo 



i r c; 



SHIPORDER 



AIRQRDER 



DIGITS 



LAUNCH 



Free Energy Surf 


Free Enemy Air 


SQ-538 


Course 4 5 


Weapons Tight 


0 8 9 


D E C M Off 


Eingo 


6 5 1 


Speed 3 2 


Refuel VK-000 


SSO-53 


Weapons Tight 


6 7 8 


5 6 4 


1 C 


Mission Strike 


Contr ol_K 


Free Enemy Sub 


Speed 4 9 


3 7 5 


9 17 


Mission Cap 


SSQ-47 


ARBOC 


Mission AEW 


16 0 


Free Enemy All 


Cover 


2 4 6 


D E C M On 


Mission Search 


8 3 6 


Course 5 4 


Mission Strkcap 


9 4 5 



KA6D FI 4 A 7 
C 6 MK4 6A 
Froceec. Pcsi 
E2C S3A 8 
A7E AGE 5 

3 2 Sidewncir 
0 4 SSO-62 
Stop 

2 1 MK 84 
F14A SF2F 7 
KA6D F14T 

4 3 SC53S 
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Configuration to Train 
Voice Patterns on 
the Verbex 5000 



VT- 102 



SPEECH UNIT 




ADM 31 




VERBEX 



** 

d 


J 0B 


ipnq: 


T 

OFF 



3000 



** 



: THE ADAPTER IS STAMPED: " VERBEX ONLY 



NOTE: THE DIAGRAM DEPICTS VIEWING THE 
MACHINES FROM THE REAR. 



FIGURE * 1 
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Configuration to Run 
NW155 on the 
Verbex 5000 



SPEECH UNIT 



VT- 102 



VT-102 




□ 


OH 


COHH 




r 

OFF 



> 



ADM 31 



OH 

* 

OFF 



□m 



WIRE 

TAGGED 

*15 



VERBEX 



NULL 

MODEM 



3000 



** 

d 


] 0¥ 


nnr 


T 

OFF 



” : THE ADAPTER IS STAMPED: “ VERBEX ONLY 

NOTE: THE DIAGRAM DEPICTS VIEWING THE 
MACHINES FROM THE REAR. 



FIGURE *2 
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NOTE; 

MICROPHONE POSITION IS CRITICAL, THE 
MICROPHONE SHOULD BE NO MORE THAN 1/2 
INCH FROM THE LIPS WITHOUT TOUCHING THEM. 
(I.E. AS SHOWN IN THE FIGURE ABOVE) 




THE MICROPHONE POSITION SHOULD BE ®32,Y? 
S&IMIfSILE LOWER THAN THE CENTER OF 
THE MOUTH . 



FIGURE *3 
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APPENDIX G 



MACINTOSH MICROCOMPUTER USERS GUIDE 



The purpose ir. accomplishing this training is to prepare to 
play various NNISS Wargair.e Scenarios as an Aircraft Carrier 
Battle Group Commander curing the course of the individual recorc 
trial. Commands for the game will be entered using the point 
select method to builc acceptable commands in the place of a 
conventional keyboard. 

To accomplish the necessary training, utilize the following 
steps to attempt to gain some skill in building commands: 

1. Reach around to the left side of the MAC, and turn it on 
using the rocker switch on the upper left side of the back. Place 
the mini-disk, label up, in the slot on the lower right of the 
front, with the metal tab entering the slot first. (NOTE: It is 
not necessary to open the tab first as it is done automatically 
by the drive.) Wait for the system to load itself (which may take 
a little over a minute) . 

2. You will see a desk top with two or more file icons. Using 
the mouse, move the arrow to the icon labeled "Offline Training", 
and with the arrow placed anywhere on the icon, click the mouse 
button twice. The program, which conditions the MAC as an MWISS 
terminal, will now load and run itself. 

3. When the load is complete you will see a series of column 
headers along the top of the desk top (i.e. "NR" contains numbers, 
etc.). At this point you are actually corresponding with a 
program written in BASIC. This program provides menus of command 
words from a limited vocabulary. It supports sequential selection 
of these words and their associated display (in an orderly fashion 
on the screen) in order to allow the build-up of full commands. 
The program also accomplishes communications with the VAX and 
returns information from the VAX to the screen; sinilar to the 
output provided by the VT100 terminal. To select a w ore from a 
menu use the mouse to position the arrow on the menu title alcng 
the top of the screen, push dov/n the mouse button, and hold it 
down. The menu will extend downward in its own window (see the 
attached example menus) . Continue to held the mouse button down 
while you move the mouse to position the arrow on the word of 
your choice. When the arrow is over each word it will be highlighted. 
To select one of the words simply let up on the button. The 
window will disappear and the v/ord will be displayed on the cesk 
top. When a command is complete, move to the right-most column 
and select "EXECUTE ORDER" from this menu. The VAX will respond 
to the order when the system is "on-line" in the future, but 
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realistic practice sessions can still be obtained in the "off-line 
mode . 



4. Notice that the right-most menu contains a selection for 
"hANUAL ENTRY". This allows you to type in the woros that are- 
otherwise not on the menus. The key under your left palm with an 
infinity sign is the control key. Use the delete key to back out 
errors in this mode. Cnee you have selected this mode , you must 
then type your desired werd and clcse the session with an 
which will take you back to mouse control. Now "EXECUTE ORDER" 
must be selected from the last menu in order to complete the command. 

5. To terminate the session, select "QUIT PROGRAM" on the 
right most menu. At the next window under the FILE menu select 
"QUIT", and under the next windov; under the FILE menu select 
"EJECT". When the mini-floppy disk is ejected, turn the machine 
off with the switch in the back. 
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NUMBER MENU 



ENCLOSURE 1 
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| « NR 


U]i*RDDR £LEK Ml<HT RfFTI RCTT2 UIPS/7KS DSG/TGT CR 




RRUIUS ... 

PI OT LOB ... 
ERASE LOB ... 

... ESM 

CENTER (on) ... 
FORCE ... 
POSITION 
TRRCK ... 

DROP ... 

010 ... 

Place GRID _ 
Place CIRCLE ... 
Place HMflRK ... 
Cancel OHIO ... 
Cancel CIRCLE ... 
Cancel HMflRK ... 
SHIFT ~ 

HELP !!! 


IISS Irmnpr OFFLINE ^ 






B 



GRAPHICS MENU 







ELEK MUMT nrm RCET2 10PS/TK5 OSG/TGT CR 



FORCE _ 



oiner OffllsE 



Constellation 
Paul „ 
Rdams „ 
Cincmattl 
BarDie _ 
Roanoke 



Goldsborough 
... MP7 ... 

... SH7 ... 

... UR7 — 

... UF7 ... 

... UIC7 ... 

... UUI 7 ... 

... TO I POINT 2 ... 



HEIPM! 



ENCLOSURE 2 



113 




ELECTRONIC MENU 



* 



NR GPH RODR EUR 



MWfF 



RCFTI RCFT2 WPS/TJCS OSG/TGT CTl 




... COURSE ... 

... SPEC 0 ... 

... STATION _ 

F oi SiJDmnrmps 
... SURERCE 
... PERISCOPE 
.... OEPTH ... 



Min( pIImupous 
... RLT1TUUE ... 

... COUER _ 

... HRNOOUER 

... PRCCEEO COURSE.- 

... PROCEEO POSITION ... 

... iuportsurhice ... 

... report Rin ... 

... Ripom siiiMnnisE ... 

... OROIRS 




HEIP III 



V 



MOVEMENT MENU 
ENCLOSURE 3 
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1 C NR GPH POOR EUH MUMT 


^RCFT2 UIFS/IKS OSG/TGT CTl 


-p — ■■ - NllMSS Tl 


... LAUNCH ... 

... R6E_ 

... E2C ... 

... FHfl ... 

... KR60 ... 

... P3C ... 

.... SH2F ... 

... LORO ... 

... MK85 ... 

... PHOENIH ... 

... SHRIKE ... 

... SPRRRGUJ... 

... SIOEIUINOER ... 
... LURltEYE ... 

... IIRRPOON ... 

... MK At ... 

... REFUEL.*. 

... BINGO TO ... 

.... STOP 
HELP !!! 






1 


B 



AIRCRAFT 1 MENU 



4 NR GPH BOOR tltK MUMT RCfTI | 
~ - O N'UJISS Trainer Cl 



a cnr 



klUPS/TKS DSG/TCT CTl 



com ^ . 

MISSION _ 
... CRP 



, RIRTRNKER 
SERRCH 
STRIKE CRP 
SURUIELLHNCE 
STRIKE 



TRKE ... 
COUER... . 
ALTITUDE ... 
STOP 



HELP ! ! ! 



AIRCRAFT2 MENU 
ENCLOSURE 4 
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C Nfl GPH RODR [UH MUMT RCfTI RCfT2 I 



IBfS/TKS 



iDSG/TGT CTl 



NliUSS IrainerOf 



lOtflPOfATlGrtT 
UJCRPONS r Rf E _ 

... RIL ... 

... Rlfl 
... SURFACE 
... SU8MRRINE 
... ENEMY 



- 



FIRE 



TAKE. 



... HRPON CRUISE 
. MK48 TORPEOO 
, RSROC TORPEOO . 

... BRO 

... BEO 

„. BPO ... 

.. BSO „ ... 

.. BUG ... 



, BERRING 
RANGE ... 
HEIP !!! 



WEAPONS/TRACK MENU 



* NR GPH RDDR ElEK MUMT RCfTI RCFT2 WPS/TKS 



NUJISS Irainer OFFLINE S 



QESIG EnLmY 
OESIG FRIENDLY ... 
0ESI6 NEUTRRL ... 
DESIG UNKNOWN ... 
RT ... 

... RLEKSKEYEU 
... PETROPRULOUSK 
... LUONSON 
... ULROIUOSTOK 



Hr i p 



DESIGNATING TARGET MENU 



ENCLOSURE 5 
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* NR GPH RDDR ELEH MUMT RCFTI RCF72 UIPS/7KS DSG/7GT g«7 



MUJiSS Trainer OFFLINE 1 



EHECUTE Order 
ConK Order 
Manual Input 
HELP!!! 

DANGER - 

do not use UuH during 
program run ! 

QUIT PROGRAM 



IE 



CONTROL MENU 



ENCLOSURE 6 
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